CN111694818A - Fund performance data checking algorithm - Google Patents

Abstract

The invention discloses a fund performance data checking algorithm, which comprises the following steps: s1, judging whether the single data is available according to the validity logic of the fund basic data; s2, for fund products with a plurality of data sources, the availability and the priority of the data sources can be set independently; s3, setting the availability and priority of data sources according to the types of fund products; s4, based on the screening logic, the single fund product performance data sequence only has a unique source at each time point; and S5, finding abnormal points in the fund performance data through a checking algorithm, and manually checking. In the field of big data, data cleaning is a key production flow for guaranteeing data quality. Based on the verification algorithm, automatic statistics and abnormal prompt of quantitative data are realized. Through the mode of man-machine combination, guarantee that fund achievement data acquisition's is complete, timely, accurate, reduce the work load that the manual work was checked, improve the treatment effeciency, optimize work flow, promote data quality.

Description

Fund performance data checking algorithm

Technical Field

The invention relates to the technical field of fund performance data verification, in particular to a calculation method of an arithmetic cumulative net value algorithm, a composite rate of return algorithm, performance data disclosure frequency statistics and the like.

Background

As a specific application of financial science and technology, a fund data verification platform deeply fuses fund data and big data technology, and based on a verification algorithm, a big data engine is used for realizing automatic statistics and exception prompt of quantitative data. And the integrity, timeliness and accuracy of fund performance data acquisition are guaranteed in a man-machine combination mode. The fund performance data checking algorithm can reduce the workload of manual checking, improve the processing efficiency, optimize the working flow and improve the data quality.

In order to meet the requirement of performance data proofreading of different types of fund products, the data structure is compatible with the data characteristics of various fund products recruited by public and private parties; the algorithm design level is subjected to unified processing, so that the development of a verification system is facilitated, and the difficulty in program implementation is reduced.

Due to the complexity of the data abnormal condition, the abnormal data points found by the verification algorithm still need to be checked and processed manually.

In the field of big data, data cleaning is an important link and is a key production flow for guaranteeing data quality. The fund performance data is easy to generate errors and omissions in the data acquisition process due to the fact that data sources are scattered and the disclosing channels are diversified. Strict screening and checking processes are required to ensure the data quality, so that the data quality has further application value.

Disclosure of Invention

Aiming at various problems which are easy to occur in the process of collecting the fund performance data, the fund performance data verification algorithm provides a mathematical model and a calculation method for realizing the development of an automatic data cleaning program. The verification algorithm can find abnormal points of data through statistics and provide basis for manual examination.

The fund performance data checking algorithm is characterized by comprising the following steps of:

s1, judging whether the single data is available according to the validity logic of the fund basic data;

s2, for fund products with a plurality of data sources, the availability and the priority of the data sources can be set independently;

s3, setting the availability and priority of data sources according to the types of fund products;

s4, based on the screening logic, the single fund product performance data sequence only has a unique source at each time point;

and S5, finding abnormal points in the fund performance data through a checking algorithm, and manually checking.

Further, in step S1, the validity logic of the fund basic data is specifically: the unit net value of the fund product is greater than 0 and is valid data, and the unit net value is null or less than or equal to 0 and is invalid data; the currency type fund product, the income is greater than 0 as valid data every ten thousand plan, null or less than or equal to 0 as invalid data; the numerical values of the dividend, the capital injection, the capital withdrawal and the split ratio are more than 0 and are effective data, and the numerical values are null or less than or equal to 0 and are ineffective data; valid data is marked as available and invalid data is marked as unavailable.

Further, in step S4, the screening logic of the fund basic data specifically includes: a single piece of data is available and a data source is available; if the available data of a plurality of sources exist in the same time point of a single fund product, sorting the available data according to the individually set priority, and selecting the source data with the highest priority; if the fund product does not independently set the data source priority, sorting the fund product according to the fund product type priority, and selecting the source data with the highest priority.

Further, in step S5, the verification algorithm specifically includes:

s5.1, checking the net value of the initial unit of the fund product: the initial unit net value is equal to 1, and is correct, otherwise, the unit net value is abnormal;

s5.2, checking the arithmetic accumulated net value:

arithmetic cumulative net worth = back complex right unit net worth + back complex right cumulative dividend-back complex right cumulative investment + back complex right cumulative investment withdrawal

It is true if the absolute value of the deviation is less than 0.0005 between the derivative of the arithmetically accumulated net value and the revealed value; otherwise, the data is abnormal, which indicates that the data such as unit net value, dividend, investment, withdrawal or split ratio has errors and omissions;

s5.3, checking the composite yield rate: calculating the composite yield between estimated value days, if the absolute value of the composite yield is smaller than the upper limit value, the composite yield is correct, otherwise, the composite yield is abnormal, and the data of unit net value, dividend, investment, withdrawal or split ratio and the like are missed;

setting the upper limit value of the composite yield absolute value as 20% according to the unilateral 90% confidence interval;

a composite rate of return algorithm:

wherein Nb represents an initial unit net value, Ne represents an ending unit net value, Ni represents an i time point unit net value, Di represents i time point division red, Ii represents i time point investment, DTi represents i time point investment withdrawal, Si represents i time point division coefficient, and R represents composite yield;

s5.4, checking the daily average composite yield: calculating the daily average composite yield between the estimated days of the single fund product according to the trading days of the Shanghai deep stock market;

if the absolute value of the average daily composite rate of return is smaller than the upper limit value, the method is correct; otherwise, the data is abnormal, which indicates that the data such as unit net value, dividend, investment, withdrawal or split ratio has errors and omissions;

setting the upper limit value of the absolute value of the daily average composite rate of return to 2% according to a single-side 90% confidence interval;

the daily average composite rate of return algorithm:

wherein R represents the composite rate of return, n represents the number of trading days between the start day and the end day, the start day is not included, and DR represents the average daily composite rate of return;

s5.5, net value disclosure frequency verification: the days between the estimated days of a single fund product is less than or equal to the revealed frequency, and the frequency is correct, otherwise, the frequency is abnormal;

the disclosed frequency value is the estimated day interval days with the highest occurrence frequency;

s5.6, net value disclosure expiration check: the number of days between the latest evaluation date and the current date of a single fund product is less than or equal to the disclosure frequency, and the frequency is correct, otherwise, the frequency is abnormal;

the disclosure frequency is taken as the number of days between which the estimation day with the highest frequency of occurrence is spaced.

The invention has the beneficial effects that:

the fund performance data verification algorithm preferentially selects reliable source data and identifies abnormal points of the data when the original collected data is cleaned. The method provides basis for manual auditing, improves processing efficiency, optimizes work flow, improves data quality and saves data cleaning cost.

Drawings

To more clearly illustrate the practice of the invention, reference will now be made to the appended drawings, which are used in describing the practice and will now be described briefly. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale. An example of a verification algorithm is illustrated with a table graph.

FIG. 1 is a schematic flow chart of steps;

FIG. 2 is a schematic diagram of arithmetic cumulative net-value verification;

FIG. 3 is a schematic diagram of a composite rate of return check;

FIG. 4 is a schematic diagram of a daily composite profitability check;

FIG. 5 is a schematic diagram of a net worth disclosure frequency check;

FIG. 6 is a schematic diagram of a net exposure expiration check.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby. It is to be noted that technical terms or technical terms used herein should have the ordinary meaning as understood by those skilled in the art to which the present invention belongs, unless otherwise specified.

Example 1

As shown in fig. 1, the fund performance data verification algorithm comprises the following steps:

s1, judging whether the single data is available according to the validity logic of the fund basic data;

s2, for fund products with a plurality of data sources, the availability and the priority of the data sources can be set independently;

s3, setting the availability and priority of data sources according to the types of fund products;

s4, based on the screening logic, the single fund product performance data sequence only has a unique source at each time point;

and S5, finding abnormal points in the fund performance data through a checking algorithm, and manually checking.

Further, in step S1, the validity logic of the fund basic data is specifically: the unit net value of the fund product is greater than 0 and is valid data, and the unit net value is null or less than or equal to 0 and is invalid data; the currency type fund product, the income is greater than 0 as valid data every ten thousand plan, null or less than or equal to 0 as invalid data; the numerical values of the dividend, the capital injection, the capital withdrawal and the split ratio are more than 0 and are effective data, and the numerical values are null or less than or equal to 0 and are ineffective data; valid data is marked as available and invalid data is marked as unavailable.

Further, in step S4, the screening logic of the fund basic data specifically includes: a single piece of data is available and a data source is available; if the available data of a plurality of sources exist in the same time point of a single fund product, sorting the available data according to the individually set priority, and selecting the source data with the highest priority; if the fund product does not independently set the data source priority, sorting the fund product according to the fund product type priority, and selecting the source data with the highest priority.

Further, in step S5, the verification algorithm specifically includes:

s5.1, checking the net value of the initial unit of the fund product: the initial unit net value is equal to 1, and is correct, otherwise, the unit net value is abnormal;

s5.2, checking the arithmetic accumulated net value:

arithmetic cumulative net worth = back complex right unit net worth + back complex right cumulative dividend-back complex right cumulative investment + back complex right cumulative investment withdrawal

It is true if the absolute value of the deviation is less than 0.0005 between the derivative of the arithmetically accumulated net value and the revealed value; otherwise, the data is abnormal, which indicates that the data such as unit net value, dividend, investment, withdrawal or split ratio has errors and omissions;

s5.3, checking the composite yield rate: calculating the composite yield between estimated value days, if the absolute value of the composite yield is smaller than the upper limit value, the composite yield is correct, otherwise, the composite yield is abnormal, and the data of unit net value, dividend, investment, withdrawal or split ratio and the like are missed;

setting the upper limit value of the composite yield absolute value as 20% according to the unilateral 90% confidence interval;

a composite rate of return algorithm:

wherein Nb represents an initial unit net value, Ne represents an ending unit net value, Ni represents an i time point unit net value, Di represents i time point division red, Ii represents i time point investment, DTi represents i time point investment withdrawal, Si represents i time point division coefficient, and R represents composite yield;

s5.4, checking the daily average composite yield: calculating the daily average composite yield between the estimated days of the single fund product according to the trading days of the Shanghai deep stock market;

if the absolute value of the average daily composite rate of return is smaller than the upper limit value, the method is correct; otherwise, the data is abnormal, which indicates that the data such as unit net value, dividend, investment, withdrawal or split ratio has errors and omissions;

setting the upper limit value of the absolute value of the daily average composite rate of return to 2% according to a single-side 90% confidence interval;

the daily average composite rate of return algorithm:

wherein R represents the composite rate of return, n represents the number of trading days between the start day and the end day, the start day is not included, and DR represents the average daily composite rate of return;

s5.5, net value disclosure frequency verification: the days between the estimated days of a single fund product is less than or equal to the revealed frequency, and the frequency is correct, otherwise, the frequency is abnormal;

the disclosed frequency value is the estimated day interval days with the highest occurrence frequency;

s5.6, net value disclosure expiration check: the number of days between the latest evaluation date and the current date of a single fund product is less than or equal to the disclosure frequency, and the frequency is correct, otherwise, the frequency is abnormal;

the disclosure frequency is taken as the number of days between which the estimation day with the highest frequency of occurrence is spaced.

Example 2

Arithmetic cumulative net worth checking:

comparing the accumulated net value disclosure value of 5 evaluation days between 19 days in 7 and 25 days in 7 and 2013 in a fund product A with the derivative value obtained according to the arithmetic accumulated net value algorithm, the absolute deviation values are all larger than 0.0005, which indicates that the unit net value disclosed by the product between five evaluation days, red data, investment data, withdrawal data or split data and the like are missed. The specific data of this example are shown in fig. 2.

Example 3

And (4) checking the composite yield:

according to a composite yield algorithm, the composite yield of a certain fund product B is-51.98% between two evaluation days from 28 th 6 th 2013 to 5 th 7 th 2013, and the absolute value of the composite yield is larger than the upper limit value (20%). The unit net value revealed between two estimation days, data such as red, capital withdrawal or splitting ratio, and the like are indicated to have errors and omissions. The specific data of this example are shown in fig. 3.

Example 4

Checking the average daily composite yield:

according to a daily average composite rate of return algorithm, the daily average composite rate of return of a certain fund product C between two evaluation days from 19 days in 7 months in 2013 to 26 days in 7 months in 2013 is 2.56%, and the absolute value of the daily average composite rate of return is greater than the upper limit (2%). The unit net value revealed between two estimation days, data such as red, capital withdrawal or splitting ratio, and the like are indicated to have errors and omissions. The specific data of this example are shown in fig. 4.

Example 5

Net-worth revealing frequency checking:

for a fund product D, 11 evaluation days between 19 days 4 and 5 days 7 and 2013 in 2013, the evaluation day interval days with the highest frequency of occurrence are 7 days, that is, the disclosure frequency is 7 days. The estimation day 2013, 7 and 5 days, the distance between the previous estimation day 2013, 6 and 21 days, and the interval is 14 days, which are more than the disclosure frequency. There may be omissions that account for the disclosed data between these two evaluation days. The specific data of this example are shown in fig. 5.

Example 6

Net worth disclosure expiration check:

and (3) a fund product E has the latest valuation date of 7-23 days in 2013, new performance data are not released after 4 days, the disclosure frequency is 1 day according to the statistics of the data published in the past, and the fact that the net value disclosure is expired for 3 days is shown. The specific data of this example are shown in fig. 6.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the algorithm features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.

Claims (4)

1. The fund performance data checking algorithm is characterized by comprising the following steps of:

s1, judging whether the single data is available according to the validity logic of the fund basic data;

s2, for fund products with a plurality of data sources, the availability and the priority of the data sources can be set independently;

s3, setting the availability and priority of data sources according to the types of fund products;

s4, based on the screening logic, the single fund product performance data sequence only has a unique source at each time point;

and S5, finding abnormal points in the fund performance data through a checking algorithm, and manually checking.

2. The fund performance data checking algorithm according to claim 1, wherein in the step S1, the validity logic of the fund basic data is specifically: the unit net value of the fund product is greater than 0 and is valid data, and the unit net value is null or less than or equal to 0 and is invalid data; the currency type fund product, the income is greater than 0 as valid data every ten thousand plan, null or less than or equal to 0 as invalid data; the numerical values of the dividend, the capital injection, the capital withdrawal and the split ratio are more than 0 and are effective data, and the numerical values are null or less than or equal to 0 and are ineffective data; valid data is marked as available and invalid data is marked as unavailable.

3. The fund performance data checking algorithm according to claim 1, wherein in the step S4, the fund basic data screening logic specifically comprises: a single piece of data is available and a data source is available; if the available data of a plurality of sources exist in the same time point of a single fund product, sorting the available data according to the individually set priority, and selecting the source data with the highest priority; if the fund product does not independently set the data source priority, sorting the fund product according to the fund product type priority, and selecting the source data with the highest priority.

4. The fund performance data checking algorithm according to claim 1, wherein in the step S5, the checking algorithm is specifically:

s5.1, checking the net value of the initial unit of the fund product: the initial unit net value is equal to 1, and is correct, otherwise, the unit net value is abnormal;

s5.2, checking the arithmetic accumulated net value:

arithmetic cumulative net worth = back complex right unit net worth + back complex right cumulative dividend-back complex right cumulative investment + back complex right cumulative investment withdrawal

It is true if the absolute value of the deviation is less than 0.0005 between the derivative of the arithmetically accumulated net value and the revealed value; otherwise, the data is abnormal, which indicates that the data such as unit net value, dividend, investment, withdrawal or split ratio has errors and omissions;

s5.3, checking the composite yield rate: calculating the composite yield between estimated value days, if the absolute value of the composite yield is smaller than the upper limit value, the composite yield is correct, otherwise, the composite yield is abnormal, and the data of unit net value, dividend, investment, withdrawal or split ratio and the like are missed;

setting the upper limit value of the composite yield absolute value as 20% according to the unilateral 90% confidence interval;

a composite rate of return algorithm:

wherein Nb represents an initial unit net value, Ne represents an ending unit net value, Ni represents an i time point unit net value, Di represents i time point division red, Ii represents i time point investment, DTi represents i time point investment withdrawal, Si represents i time point division coefficient, and R represents composite yield;

s5.4, checking the daily average composite yield: calculating the daily average composite yield between the estimated days of the single fund product according to the trading days of the Shanghai deep stock market;

if the absolute value of the average daily composite rate of return is smaller than the upper limit value, the method is correct; otherwise, the data is abnormal, which indicates that the data such as unit net value, dividend, investment, withdrawal or split ratio has errors and omissions;

setting the upper limit value of the absolute value of the daily average composite rate of return to 2% according to a single-side 90% confidence interval;

the daily average composite rate of return algorithm:

wherein R represents the composite rate of return, n represents the number of trading days between the start day and the end day, the start day is not included, and DR represents the average daily composite rate of return;

s5.5, net value disclosure frequency verification: the days between the estimated days of a single fund product is less than or equal to the revealed frequency, and the frequency is correct, otherwise, the frequency is abnormal;

the disclosed frequency value is the estimated day interval days with the highest occurrence frequency;

s5.6, net value disclosure expiration check: the number of days between the latest evaluation date and the current date of a single fund product is less than or equal to the disclosure frequency, and the frequency is correct, otherwise, the frequency is abnormal;

the disclosure frequency is taken as the number of days between which the estimation day with the highest frequency of occurrence is spaced.

Images