CN118245316A - Foreground data monitoring method and system of three-party data information platform - Google Patents
Foreground data monitoring method and system of three-party data information platform Download PDFInfo
- Publication number
- CN118245316A CN118245316A CN202311450380.4A CN202311450380A CN118245316A CN 118245316 A CN118245316 A CN 118245316A CN 202311450380 A CN202311450380 A CN 202311450380A CN 118245316 A CN118245316 A CN 118245316A
- Authority
- CN
- China
- Prior art keywords
- data source
- data
- sub
- division
- source sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000000007 visual effect Effects 0.000 claims abstract description 118
- 230000008859 change Effects 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000007246 mechanism Effects 0.000 claims description 47
- 230000002159 abnormal effect Effects 0.000 claims description 35
- 230000001186 cumulative effect Effects 0.000 claims description 20
- 239000013598 vector Substances 0.000 claims description 20
- 230000005856 abnormality Effects 0.000 claims description 14
- 238000012800 visualization Methods 0.000 claims description 9
- 238000013079 data visualisation Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 3
- 230000008520 organization Effects 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 description 10
- 238000013461 design Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a foreground data monitoring method and a foreground data monitoring system of a three-party data information platform, comprising the following steps: collecting data source sub-division of foreground data of a three-party data platform; calculating index values of the data source sub-components under five-dimension indexes; based on the index value under the five-dimension index, carrying out visual processing on the five-dimension index and carrying out visual display; based on the visual display result, taking a preset unit as a filter, acquiring the change information of the data source sub-division, and positioning accident data according to the change information; according to the visual display result, accident data are efficiently and accurately positioned, so that the problems can be found out in advance and solved in a controllable time range, and normal and stable operation of each service line is ensured.
Description
Technical Field
The invention relates to the technical field of data information monitoring, in particular to a foreground data monitoring method and a foreground data monitoring system of a three-party data information platform.
Background
Along with the continuous deepening of knowledge of various industries on big data, the strategic significance and importance of the big data are continuously shown, and the strategic completion degree is greatly dependent on the stability of products. Along with the gradual manifestation of the diversification of products, the traditional database monitoring can not discover potential problems in the system operation process in time, database data loss is easy to cause, the influence is brought to enterprises, and the work of operation and maintenance personnel is increased in a complex and multiple way, so that the problems of incomplete monitoring, low efficiency and the like are caused. In order to better consolidate the stability of the product, relevant service data are required to be imported into a visual monitoring system, and corresponding conclusions are obtained through different indexes and data, so that the problems are found out in advance and are solved in a controllable time range.
Disclosure of Invention
The invention provides a foreground data monitoring method and a foreground data monitoring system for a three-party data information platform, which are convenient for finding out problems in advance and solving the problems by analyzing different indexes and data and effectively positioning accident data in time.
A foreground data monitoring method of a three-party data information platform comprises the following steps:
S1: collecting data source sub-division of foreground data of a three-party data platform;
S2: calculating index values of the data source sub-components under five-dimension indexes;
The five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
s3: based on the index value under the five-dimension index, carrying out visual processing on the five-dimension index and carrying out visual display;
Comprising the following steps: dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-dimension index values;
S4: based on the visual display result, the change information of the data source sub-components is acquired by taking a preset unit as a filter, and accident data is positioned according to the change information.
After S1, further comprising:
performing traversal analysis on the data source sub-division to extract the data source sub-division with abnormality;
Mapping the data source sub-division with the abnormality to obtain the normal data source sub-division.
S2, calculating the index value of the data source sub-division under the index of five large dimensions comprises the following steps:
the five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
The grading calling quantity is the sum value of all calls of each client mechanism to different products and versions, the grading is divided into the average value of normally graded data source sub-scores, the grading point is the minimum value, the 5% point, the 10% point, the 25% point, the 50% point, the 75% point, the 90% point and the maximum value of the normally graded data source sub-scores, the grading cumulative distribution is the probability distribution of the sub-scores effectively called by each data source, and the grading group stability index is the index for measuring the deviation between the actual sub-score and the preset sub-score.
S3, performing visualization processing on the five-dimension index based on the index value under the five-dimension index, and performing visualization display, wherein the method comprises the following steps:
Dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-large-dimension index values.
Performing visualization processing on the five-dimension index, performing visualization display, and further comprising:
Acquiring a first mechanism feature of the first visual display, and determining a first product feature under the first mechanism feature;
acquiring a second customer characteristic of the second visual display, and determining a second mechanism characteristic under the second customer characteristic;
performing first matching on the first mechanism characteristic and the second mechanism characteristic to obtain a first matching result;
Performing second matching on the first product characteristics and the second product characteristics to obtain a second matching result;
And fusing the first visual display and the second visual display based on the first matching result and the second matching result to obtain a comprehensive visual display.
The construction mode of the display frame for comprehensive visual display is as follows:
Analyzing and dividing index values of the data source sub-components under five-dimension indexes according to client institutions and product versions respectively to obtain a first value set and a second value set;
first grouping the first value set based on the attribute of the client mechanism, and second grouping the first value set according to the time unit;
Third grouping the second value set based on the attribute of the product version, and fourth grouping the second value set according to the time unit;
Respectively determining a visual display format corresponding to the five-dimension preset index, and determining a visual algorithm corresponding to the visual display format;
compiling the first group, the second group, the third group and the fourth group by utilizing the visualization algorithm to obtain data visualization information;
acquiring position information of each region position in the visual display interface;
Acquiring a display template from a template library, and comprehensively matching the data visualization information and the position information based on the display template to obtain a plurality of groups of configuration resources;
according to the visual requirement of a user, newly selecting target information from the data visual information, and matching the target information with the position information to obtain target visual display information;
Selecting a target configuration resource matched with the target visual display information from the plurality of groups of configuration resources;
And constructing the display frame of the comprehensive visual display by utilizing the target configuration resource.
S4, based on the visual display result, taking a preset unit as a filter to acquire the change information of the data source sub-components, wherein the method comprises the following steps:
acquiring a monitoring demand of a user, determining a primary classification name and a secondary classification name according to the monitoring demand, and screening from the visual display result based on the primary classification name and the secondary classification name to obtain an initial display result;
Analyzing the monitoring requirement and determining a unit index of the preset unit;
Verifying the primary classification name and the secondary classification name according to a three-party data platform unit, a product service coding unit, a product version unit and a client organization unit in the unit index, and judging whether the initial display result meets the monitoring requirement or not;
if so, carrying out position marking on the initial display result based on the unit index, and carrying out screener marking on the position marked part according to the position marking result to obtain a screener marking result;
otherwise, the primary classification name and the secondary classification name are determined again;
dynamically adjusting the initial display result according to the position marking result and the screener marking result to obtain a target display result;
Comparing the display results in the same display format based on the time units and the call volume units in the unit indexes of the target display results to obtain corresponding comparison results, and acquiring local change information of the data source sub-components according to the comparison results;
carrying out peer-to-peer comparison on comparison results under different display formats to obtain change relation information among various parts of the data source sub-division;
and obtaining the change information of the data source sub-division based on the local change information and the change relation information.
S4, positioning accident data according to the change information, wherein the method comprises the following steps:
according to the change information, obtaining an abnormal display result corresponding to the change amplitude larger than a preset amplitude in the visual display result;
Acquiring the time length of the abnormal display result, and acquiring a target data source sub-score corresponding to the abnormal display result;
Dividing the target data source sub-division into a current data source sub-division and a historical data source sub-division according to the time length;
Obtaining a child abnormal value of the target data source child based on the values of the current data source child and the historical data source child in five dimensions;
judging whether the child abnormal value is larger than a preset child abnormal value or not;
if yes, determining that the current data source sub-molecule has abnormality;
Otherwise, determining that the current data source sub-division is normal;
After determining that the current data source sub-division has abnormality, acquiring a current data source corresponding to the current data source sub-division, and acquiring a historical data source corresponding to the historical data source sub-division;
And determining whether the current data source is abnormal or not based on the current data source and the historical data source, and if so, positioning the current data source to serve as accident data.
Determining whether the current data source is abnormal based on the current data source and the historical data source, if so, positioning the current data source as accident data, wherein the method comprises the following steps:
Dividing the current data source according to preset data dimensions to obtain first multi-dimensional data, and constructing a first vector according to the first multi-dimensional data;
Dividing the historical data source according to preset data dimensions to obtain second multidimensional data, and constructing a second vector according to the second multidimensional data;
take the value on the basis of the similarity between said first vector and second vector;
if the similarity value is smaller than the preset similarity;
Determining that the current data source is abnormal, and positioning the current data source to serve as accident data;
otherwise, determining that the current data source is normal.
A foreground data monitoring system of a three-party data information platform, comprising:
the acquisition module is used for acquiring data source sub-divisions of the foreground data of the three-party data platform;
The dimension index module is used for calculating index values of the data source sub-components under five dimension indexes;
The five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
the visual display module is used for performing visual processing on the five-dimension index based on the index value under the five-dimension index and performing visual display;
Comprising the following steps: dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-dimension index values;
the abnormal positioning module is used for acquiring the change information of the data source sub-components by taking a preset unit as a filter based on the visual display result and positioning accident data according to the change information.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flowchart of a foreground data monitoring method of a three-party data information platform according to an embodiment of the present invention;
FIG. 2 is an exemplary diagram of a visual legend in accordance with an embodiment of the present invention;
FIG. 3 is an exemplary diagram of a visual report in an embodiment of the present invention;
fig. 4 is a block diagram of a foreground data monitoring system of a three-party data information platform according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
Example 1
The embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, as shown in fig. 1, comprising the following steps:
S1: collecting data source sub-division of foreground data of a three-party data platform;
S2: calculating index values of the data source sub-components under five-dimension indexes;
The five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
s3: based on the index value under the five-dimension index, carrying out visual processing on the five-dimension index and carrying out visual display;
Comprising the following steps: dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-dimension index values;
S4: based on the visual display result, the change information of the data source sub-components is acquired by taking a preset unit as a filter, and accident data is positioned according to the change information.
In this embodiment, the preset units include a time unit, a three-party data platform unit, a product service encoding unit, a product version unit, a customer institution unit, and a call volume unit.
In this embodiment, the positioning incident data is determined from the five-dimensional index value changes for different customers using different products over different time periods.
In this embodiment, the five-dimensional index includes a score call amount, a score average, a score locus, a score cumulative distribution, and a score population stability index.
In this embodiment, the foreground data is foreground data corresponding to different customer institutions, scoring products, model versions, and time point information.
The beneficial effects of above-mentioned design scheme are: the data source sub-division is acquired through the three-party data platform, the data source sub-division is visually displayed from five-dimension indexes, the data are tidied, the data monitoring efficiency is improved, accident data are efficiently and accurately positioned according to visual display results, the problem can be found out in advance and solved in a controllable time range, the technical indexes of calling the three-party data by different subdivision and data version can be distinguished through real-time monitoring of the three-party data information in each dimension, the omnibearing, multi-angle and cross-time monitoring is carried out, the change of the data source production call can be known in time, and the normal and stable operation of each service line is ensured.
Example 2
Based on embodiment 1, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, and after step 1, the foreground data monitoring method further comprises:
performing traversal analysis on the data source sub-division to extract the data source sub-division with abnormality;
Mapping the data source sub-division with the abnormality to obtain the normal data source sub-division.
In this embodiment, the active data source sub-score is naturally at most in a large number of special cases other than normal scoring, such as scoring timeout, guest group miss, etc., so all special cases need to be filled in with a specified value.
The beneficial effects of above-mentioned design scheme are: the data source sub-division with abnormality is supplemented, so that the integrity and the accuracy of the data source sub-division are ensured, and the quality of the data source sub-division is ensured.
Example 3
Based on embodiment 1, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, and in S2, calculating index values of the data source sub-components under five-dimension indexes comprises:
the five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
The grading calling quantity is the sum value of all calls of each client mechanism to different products and versions, the grading is divided into the average value of normally graded data source sub-scores, the grading point is the minimum value, the 5% point, the 10% point, the 25% point, the 50% point, the 75% point, the 90% point and the maximum value of the normally graded data source sub-scores, the grading cumulative distribution is the probability distribution of the sub-scores effectively called by each data source, and the grading group stability index is the index for measuring the deviation between the actual sub-score and the preset sub-score.
In this embodiment, the calculation formula of the scoring points is as follows:
P＝(n/N)*100％
Wherein P represents the values of the scoring points, N represents the positions of the values ordered according to the order of magnitude, and N represents the number of the data source sub-points.
In this embodiment, the calculation formula of the score group stability index is as follows:
Wherein S 0 represents the preset sub-division, N represents the number of the data source sub-division, and S i represents the actual sub-division of the ith data source sub-division.
The beneficial effects of above-mentioned design scheme are: and further analyzing the data source sub-components according to five-dimension indexes to determine the values of the data source sub-components in all dimensions, thereby providing a basis for determining accident data.
Example 4
Based on embodiment 1, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, as shown in fig. 4, in S3, based on the index value under the five-dimension index, performing visualization processing on the five-dimension index, and performing visualization display, including:
Dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-large-dimension index values.
In this embodiment, the first part is a visual diagram such as that shown in fig. 2, and the second part is a visual report such as that shown in fig. 3. The fonts in fig. 2 and 3 are microsoft black, fig. 2 shows a histogram of call volume for each customer mechanism using the selected sub data source at label 1, a score line for each customer mechanism using the selected sub data source at label 2 of fig. 2 using a 146,195,146 color RBG, a score area for each customer mechanism using the selected sub data source at label 3 of fig. 2 using a 89,151,113、103,161,122、115,172,129、142,192,135、167,202,136、192,209,137、220,215,137、226,101,101、126,182,133、247,223,147, color RBG from top to bottom, a 125,186,137, 194,131,120, 89,151,113, 103,161,122, 115,172,129, 142,192,135 from top to bottom using a RBG color RBG, a score cumulative distribution map for each customer mechanism using the selected sub data source at label 4 of fig. 2 using a 89,151,113, 103,161,122, 342, 142,192,135, 167,202,136, 65,136,93, 63,135,92 from top to bottom using a RBG color RBG, a call date, cumulative volume, call volume, score, uniform score, and score of each product version using the selected sub data source at label 1 of fig. 3 using the selected sub data source, and a score of 255,255 to bottom using the RBG color RBG score, 255.
The beneficial effects of above-mentioned design scheme are: the method comprises the steps of dividing the data source sub-division according to the client mechanism to obtain the calling quantity of a plurality of groups of sub-data source sub-division, and performing first visual display, so that the change of the production calling of the data source of the client mechanism can be clearly displayed, the operation information of each client mechanism can be visually determined, the plurality of groups of sub-data source sub-division is divided through the product version, the second visual display is performed by utilizing the five-dimension index value, the value change of the product version under the five-dimension is clearly determined, the operation information of each product version is visually determined, the efficiency of data monitoring is improved, and the accident data can be conveniently and accurately positioned in a high-efficient manner by performing omnibearing, multi-angle and cross-time visual display.
Example 5
Based on embodiment 4, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, which performs visual processing on the five-dimensional index and performs visual display, and further comprises:
Acquiring a first mechanism feature of the first visual display, and determining a first product feature under the first mechanism feature;
acquiring a second customer characteristic of the second visual display, and determining a second mechanism characteristic under the second customer characteristic;
performing first matching on the first mechanism characteristic and the second mechanism characteristic to obtain a first matching result;
Performing second matching on the first product characteristics and the second product characteristics to obtain a second matching result;
And fusing the first visual display and the second visual display based on the first matching result and the second matching result to obtain a comprehensive visual display.
In this embodiment, the integrated visual presentation may present sub-variations in the data sources for different customers using different products.
The beneficial effects of above-mentioned design scheme are: through carrying out comprehensive visual display with customer mechanism and product version, can show that different customers use the data source sub-minute of different products to change, synthesize customer mechanism and product version and show, the show content is finer, is convenient for high-efficient accurate location accident data.
Example 6
Based on the embodiment 5, the foreground data monitoring method of the three-party data information platform is characterized in that the construction mode of the comprehensive visual display frame is as follows:
Analyzing and dividing index values of the data source sub-components under five-dimension indexes according to client institutions and product versions respectively to obtain a first value set and a second value set;
first grouping the first value set based on the attribute of the client mechanism, and second grouping the first value set according to the time unit;
Third grouping the second value set based on the attribute of the product version, and fourth grouping the second value set according to the time unit;
Respectively determining a visual display format corresponding to the five-dimension preset index, and determining a visual algorithm corresponding to the visual display format;
compiling the first group, the second group, the third group and the fourth group by utilizing the visualization algorithm to obtain data visualization information;
acquiring position information of each region position in the visual display interface;
Acquiring a display template from a template library, and comprehensively matching the data visualization information and the position information based on the display template to obtain a plurality of groups of configuration resources;
according to the visual requirement of a user, newly selecting target information from the data visual information, and matching the target information with the position information to obtain target visual display information;
Selecting a target configuration resource matched with the target visual display information from the plurality of groups of configuration resources;
And constructing the display frame of the comprehensive visual display by utilizing the target configuration resource.
In this embodiment, the index values of the data source sub-components corresponding to the client mechanism under the index of five dimensions form a first value set, and the index values of the data source sub-components corresponding to the product version under the index of five dimensions form a second value set.
In this embodiment, the first and second packets differ in packet dimension, the first packet being classified from the category to which the client mechanism belongs, the second packet being classified according to the time of acquisition of the data source of the client mechanism, as are the third and fourth packets.
In this embodiment, the visual information includes in what format each packet is displayed, such as a graph, bar graph, etc., as well as the color, size, etc., of the display.
In this embodiment, the display template includes a determination of a display format, a determination of a position, and a determination of each area of the visual presentation interface, and the configuration resource is a resource required for implementing the sushi.
The beneficial effects of above-mentioned design scheme are: the method has the advantages that index values under five-dimension indexes are analyzed and divided according to the data source sub-division, the regularity of visual display is guaranteed, the viewing and searching of a user are facilitated, the data visual information is obtained through grouping and compiling according to a visual algorithm, the display formats with rich types are provided, the requirements of designing various visual display interfaces are met, the data visual information and the position information are comprehensively matched by taking a display template as a reference, multiple groups of configuration resources are obtained, a resource basis is provided for the display of the visual display interfaces, the client mechanism and the product version are comprehensively displayed, the display content is more refined, and the accident data is conveniently and efficiently and accurately positioned.
Example 7
Based on embodiment 1, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, in S4, based on a visual display result, the foreground data monitoring method takes a preset unit as a filter to obtain the change information of the data source sub-components, and the foreground data monitoring method comprises the following steps:
acquiring a monitoring demand of a user, determining a primary classification name and a secondary classification name according to the monitoring demand, and screening from the visual display result based on the primary classification name and the secondary classification name to obtain an initial display result;
Analyzing the monitoring requirement and determining a unit index of the preset unit;
Verifying the primary classification name and the secondary classification name according to a three-party data platform unit, a product service coding unit, a product version unit and a client organization unit in the unit index, and judging whether the initial display result meets the monitoring requirement or not;
if so, carrying out position marking on the initial display result based on the unit index, and carrying out screener marking on the position marked part according to the position marking result to obtain a screener marking result;
otherwise, the primary classification name and the secondary classification name are determined again;
dynamically adjusting the initial display result according to the position marking result and the screener marking result to obtain a target display result;
Comparing the display results in the same display format based on the time units and the call volume units in the unit indexes of the target display results to obtain corresponding comparison results, and acquiring local change information of the data source sub-components according to the comparison results;
carrying out peer-to-peer comparison on comparison results under different display formats to obtain change relation information among various parts of the data source sub-division;
and obtaining the change information of the data source sub-division based on the local change information and the change relation information.
In this embodiment, the monitoring requirement is, for example, a change of a data source of a certain customer mechanism, and at this time, the corresponding initial display result is first classified by the certain customer mechanism as a first class, and then subdivided by the certain customer mechanism as a second class according to a product version, whereas if a change of a data source of a certain product version is required, at this time, the corresponding initial display result is first classified by the certain product version as a first class, and then subdivided by the certain product version as a second class according to the customer mechanism.
In this embodiment, the unit index is, for example, a unit index of a time unit is month, a unit index of a three-party data platform unit is a certain platform, a product version unit index is a certain product type, and the like.
In this embodiment, the position marking of the initial display result is performed based on the unit index, and the position of each display graph in the initial display result is marked with the corresponding unit index.
In this embodiment, the filter marking at the position mark is used to determine the accuracy, content, etc. that the initial display result should be displayed according to the unit index.
In this embodiment, the local change information is a comparison between the time unit and the call volume unit of the data source sub-partition in the unit index, the change relation information between the local parts is that after the local change information is obtained, the unit conversion is performed according to the difference between the time unit and the call volume unit of the local change information, and then the comparison is performed, and then the comparison process should be considered in the three-party data platform unit, the product service coding unit, the product version unit and the client mechanism unit corresponding to the comparison result under different display formats.
The beneficial effects of above-mentioned design scheme are: the initial display result is obtained from the visual display result according to the monitoring requirement of the user, then the unit index of the screener is determined according to the unit index of the preset unit of the monitoring requirement, the target display result is obtained, the unit selection of the target display result is ensured to be more in line with the monitoring requirement, the change information of the obtained data source sub-component is more accurate, and an accurate basis is provided for determining accident data.
Example 8
Based on embodiment 1, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, in S4, positioning accident data according to the change information, including:
according to the change information, obtaining an abnormal display result corresponding to the change amplitude larger than a preset amplitude in the visual display result;
Acquiring the time length of the abnormal display result, and acquiring a target data source sub-score corresponding to the abnormal display result;
Dividing the target data source sub-division into a current data source sub-division and a historical data source sub-division according to the time length;
Obtaining a child abnormal value of the current data source child based on the values of the current data source child and the historical data source child in the fifth dimension;
the calculation formula of the sub-division abnormal value F is as follows:
wherein N N represents the number of data sources corresponding to the current data source sub-division, N D represents the number of data sources corresponding to the historical data source sub-division, N i represents the value of the ith dimension index of the current data source sub-division, D i represents the value of the ith dimension index of the historical data source sub-division, τ i represents the influence weight of the ith dimension index on the abnormality of the ith dimension index, and the value is (0.5, 1), i=1, 2,3,4,5; when i=1, the score call quantity index is valued, when i=2, the score equipartition index is valued, when i=3, the score point index is valued, when i=4, the score cumulative distribution index is valued, and when i=5, the score cumulative distribution index is valued;
judging whether the child abnormal value is larger than a preset child abnormal value or not;
if yes, determining that the current data source sub-molecule has abnormality;
Otherwise, determining that the current data source sub-division is normal;
After determining that the current data source sub-division has abnormality, acquiring a current data source corresponding to the current data source sub-division, and acquiring a historical data source corresponding to the historical data source sub-division;
And determining whether the current data source is abnormal or not based on the current data source and the historical data source, and if so, positioning the current data source to serve as accident data.
In this embodiment, the current data source sub-score corresponds to the time length, and the historical data source sub-score corresponds to the time length.
In this embodiment, the child-division anomaly value is considered from five-dimensional indexes, and the anomaly value is determined according to the difference of the five-dimensional index values between the current data source child-division and the historical data source child-division, so as to determine whether the child-division anomaly exists.
In this embodiment, the dimension index has an influence weight on the score anomaly, for example, the score adjustment amount index is 0.8, the score average index is 0.7, the score locus index is 0.6, the score cumulative distribution index is 0.6, and the score cumulative distribution index is 0.9.
In this embodiment, the historical data source child represents a child that the data source normally obtained.
The beneficial effects of above-mentioned design scheme are: the data source sub-division corresponding to the abnormal display result is analyzed to determine whether the data source is abnormal or not, and after the sub-division is abnormal, the data source is further determined whether the data source is abnormal or not according to the data source, and the accuracy of analyzing the data source and the accuracy of positioning accident data are ensured by a layer-by-layer progressive deep analysis method.
Example 9
Based on embodiment 8, the embodiment of the invention provides a foreground data monitoring method of a three-party data information platform, which determines whether the current data source is abnormal or not based on the current data source and the historical data source, and if so, locates the current data source as accident data, and comprises the following steps:
Dividing the current data source according to preset data dimensions to obtain first multi-dimensional data, and constructing a first vector according to the first multi-dimensional data;
Dividing the historical data source according to preset data dimensions to obtain second multidimensional data, and constructing a second vector according to the second multidimensional data;
take the value on the basis of the similarity between said first vector and second vector;
The calculation formula of the similarity value S is as follows:
Wherein a represents the integrated eigenvalue of the first vector, B represents the integrated eigenvalue of the second vector, n represents the number of preset dimensions, α j represents the number of vectors in the first vector corresponding to the j-th preset dimension, β j represents the number of vectors in the second vector corresponding to the j-th preset dimension, a j represents the dimensional eigenvalue of the j-th preset dimension in the first vector, and B j represents the dimensional eigenvalue of the j-th preset dimension in the second vector;
if the similarity value is smaller than the preset similarity;
Determining that the current data source is abnormal, and positioning the current data source to serve as accident data;
otherwise, determining that the current data source is normal.
In this embodiment, the preset dimensions include a user dimension, an operation dimension, a product dimension, and a market dimension.
In this embodiment, the value of the integrated characteristic value is (1, 10), which is determined by the current data source or the historical data source.
In this embodiment, the values of the dimension characteristic values are (0, 1), and are determined by the corresponding data sources in the preset dimension.
The beneficial effects of above-mentioned design scheme are: and calculating and determining the similarity between the current data source and the historical data source through the characteristic difference between the current data source and the historical data source in different preset dimensions and the characteristic difference of the first-level integral data source, so that whether the current data source is accident data or not is further determined, and the accuracy of positioning the accident data is improved.
Example 10
As shown in fig. 4, a foreground data monitoring system of a three-party data information platform includes:
the acquisition module is used for acquiring data source sub-divisions of the foreground data of the three-party data platform;
The dimension index module is used for calculating index values of the data source sub-components under five dimension indexes;
The five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
the visual display module is used for performing visual processing on the five-dimension index based on the index value under the five-dimension index and performing visual display;
Comprising the following steps: dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-dimension index values;
the abnormal positioning module is used for acquiring the change information of the data source sub-components by taking a preset unit as a filter based on the visual display result and positioning accident data according to the change information.
In this embodiment, the preset units include a time unit, a three-party data platform unit, a product service encoding unit, a product version unit, a customer institution unit, and a call volume unit.
In this embodiment, the positioning incident data is determined from the five-dimensional index value changes for different customers using different products over different time periods.
In this embodiment, the five-dimensional index includes a score call amount, a score average, a score locus, a score cumulative distribution, and a score population stability index.
In this embodiment, the foreground data is foreground data corresponding to different customer institutions, scoring products, model versions, and time point information.
The beneficial effects of above-mentioned design scheme are: the data source sub-division is acquired through the three-party data platform, the data source sub-division is visually displayed from five-dimension indexes, the data are tidied, the data monitoring efficiency is improved, accident data are efficiently and accurately positioned according to visual display results, the problem can be found out in advance and solved in a controllable time range, the technical indexes of calling the three-party data by different subdivision and data version can be distinguished through real-time monitoring of the three-party data information in each dimension, the omnibearing, multi-angle and cross-time monitoring is carried out, the change of the data source production call can be known in time, and the normal and stable operation of each service line is ensured.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. The foreground data monitoring method of the three-party data information platform is characterized by comprising the following steps of:
S1: collecting data source sub-division of foreground data of a three-party data platform;
S2: calculating index values of the data source sub-components under five-dimension indexes;
The five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
s3: based on the index value under the five-dimension index, carrying out visual processing on the five-dimension index and carrying out visual display;
Comprising the following steps: dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-dimension index values;
S4: based on the visual display result, the change information of the data source sub-components is acquired by taking a preset unit as a filter, and accident data is positioned according to the change information.
2. The foreground data monitoring method of a three-party data information platform according to claim 1, further comprising, after S1:
performing traversal analysis on the data source sub-division to extract the data source sub-division with abnormality;
Mapping the data source sub-division with the abnormality to obtain the normal data source sub-division.
3. The foreground data monitoring method of a three-party data information platform according to claim 1, wherein in S2, calculating the index value of the data source sub-division under the index of five dimensions comprises:
the five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
The grading calling quantity is the sum value of all calls of each client mechanism to different products and versions, the grading is divided into the average value of normally graded data source sub-scores, the grading point is the minimum value, the 5% point, the 10% point, the 25% point, the 50% point, the 75% point, the 90% point and the maximum value of the normally graded data source sub-scores, the grading cumulative distribution is the probability distribution of the sub-scores effectively called by each data source, and the grading group stability index is the index for measuring the deviation between the actual sub-score and the preset sub-score.
4. The foreground data monitoring method of the three-party data information platform according to claim 1, wherein the visualized processing is performed on the five-dimensional index, and the visualized display is performed, further comprising:
Acquiring a first mechanism feature of the first visual display, and determining a first product feature under the first mechanism feature;
acquiring a second customer characteristic of the second visual display, and determining a second mechanism characteristic under the second customer characteristic;
performing first matching on the first mechanism characteristic and the second mechanism characteristic to obtain a first matching result;
Performing second matching on the first product characteristics and the second product characteristics to obtain a second matching result;
And fusing the first visual display and the second visual display based on the first matching result and the second matching result to obtain a comprehensive visual display.
5. The foreground data monitoring method of the three-party data information platform according to claim 4, wherein the display framework for comprehensive visual display is constructed as follows:
Analyzing and dividing index values of the data source sub-components under five-dimension indexes according to client institutions and product versions respectively to obtain a first value set and a second value set;
first grouping the first value set based on the attribute of the client mechanism, and second grouping the first value set according to the time unit;
Third grouping the second value set based on the attribute of the product version, and fourth grouping the second value set according to the time unit;
Respectively determining a visual display format corresponding to the five-dimension preset index, and determining a visual algorithm corresponding to the visual display format;
compiling the first group, the second group, the third group and the fourth group by utilizing the visualization algorithm to obtain data visualization information;
acquiring position information of each region position in the visual display interface;
Acquiring a display template from a template library, and comprehensively matching the data visualization information and the position information based on the display template to obtain a plurality of groups of configuration resources;
according to the visual requirement of a user, newly selecting target information from the data visual information, and matching the target information with the position information to obtain target visual display information;
Selecting a target configuration resource matched with the target visual display information from the plurality of groups of configuration resources;
And constructing the display frame of the comprehensive visual display by utilizing the target configuration resource.
6. The foreground data monitoring method of the three-party data information platform according to claim 1, wherein in S4, based on the visual display result, the foreground data monitoring method uses a preset unit as a filter to obtain the change information of the data source sub-components, and the foreground data monitoring method comprises the following steps:
acquiring a monitoring demand of a user, determining a primary classification name and a secondary classification name according to the monitoring demand, and screening from the visual display result based on the primary classification name and the secondary classification name to obtain an initial display result;
Analyzing the monitoring requirement and determining a unit index of the preset unit;
Verifying the primary classification name and the secondary classification name according to a three-party data platform unit, a product service coding unit, a product version unit and a client organization unit in the unit index, and judging whether the initial display result meets the monitoring requirement or not;
if so, carrying out position marking on the initial display result based on the unit index, and carrying out screener marking on the position marked part according to the position marking result to obtain a screener marking result;
otherwise, the primary classification name and the secondary classification name are determined again;
dynamically adjusting the initial display result according to the position marking result and the screener marking result to obtain a target display result;
Comparing the display results in the same display format based on the time units and the call volume units in the unit indexes of the target display results to obtain corresponding comparison results, and acquiring local change information of the data source sub-components according to the comparison results;
carrying out peer-to-peer comparison on comparison results under different display formats to obtain change relation information among various parts of the data source sub-division;
and obtaining the change information of the data source sub-division based on the local change information and the change relation information.
7. The foreground data monitoring method of a three-party data information platform according to claim 1, wherein in S4, locating accident data according to the change information comprises:
according to the change information, obtaining an abnormal display result corresponding to the change amplitude larger than a preset amplitude in the visual display result;
Acquiring the time length of the abnormal display result, and acquiring a target data source sub-score corresponding to the abnormal display result;
Dividing the target data source sub-division into a current data source sub-division and a historical data source sub-division according to the time length;
Obtaining a child abnormal value of the target data source child based on the values of the current data source child and the historical data source child in five dimensions;
judging whether the child abnormal value is larger than a preset child abnormal value or not;
if yes, determining that the current data source sub-molecule has abnormality;
Otherwise, determining that the current data source sub-division is normal;
After determining that the current data source sub-division has abnormality, acquiring a current data source corresponding to the current data source sub-division, and acquiring a historical data source corresponding to the historical data source sub-division;
And determining whether the current data source is abnormal or not based on the current data source and the historical data source, and if so, positioning the current data source to serve as accident data.
8. The foreground data monitoring method of a three-party data information platform according to claim 7, wherein determining whether the current data source is abnormal based on the current data source and the historical data source, if so, locating the current data source as accident data comprises:
Dividing the current data source according to preset data dimensions to obtain first multi-dimensional data, and constructing a first vector according to the first multi-dimensional data;
Dividing the historical data source according to preset data dimensions to obtain second multidimensional data, and constructing a second vector according to the second multidimensional data;
take the value on the basis of the similarity between said first vector and second vector;
if the similarity value is smaller than the preset similarity;
Determining that the current data source is abnormal, and positioning the current data source to serve as accident data;
otherwise, determining that the current data source is normal.
9. The foreground data monitoring system of the three-party data information platform is characterized by comprising:
the acquisition module is used for acquiring data source sub-divisions of the foreground data of the three-party data platform;
The dimension index module is used for calculating index values of the data source sub-components under five dimension indexes;
The five-dimension index comprises a scoring call quantity, scoring equipartition, scoring parting points, scoring cumulative distribution and scoring population stability index;
the visual display module is used for performing visual processing on the five-dimension index based on the index value under the five-dimension index and performing visual display;
Comprising the following steps: dividing the data source sub-division according to a client mechanism to obtain a plurality of groups of first sub-data source sub-division, obtaining five-large-dimension index values of the plurality of groups of first sub-data source sub-division, and carrying out first visual display based on the five-large-dimension index values;
Dividing the data source sub-division according to the product version to obtain a plurality of groups of second sub-data source sub-division, obtaining five-dimension index values of the plurality of groups of second sub-data source sub-division, and carrying out second visual display based on the five-dimension index values;
the abnormal positioning module is used for acquiring the change information of the data source sub-components by taking a preset unit as a filter based on the visual display result and positioning accident data according to the change information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311450380.4A CN118245316A (en) | 2023-11-03 | 2023-11-03 | Foreground data monitoring method and system of three-party data information platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311450380.4A CN118245316A (en) | 2023-11-03 | 2023-11-03 | Foreground data monitoring method and system of three-party data information platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118245316A true CN118245316A (en) | 2024-06-25 |
Family
ID=91555435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311450380.4A Pending CN118245316A (en) | 2023-11-03 | 2023-11-03 | Foreground data monitoring method and system of three-party data information platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118245316A (en) |
-
2023
- 2023-11-03 CN CN202311450380.4A patent/CN118245316A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230079109A1 (en) | Systems and Methods for Ranking Data Visualizations | |
| US10037305B2 (en) | Computer-implemented systems and methods for time series exploration | |
| US9424318B2 (en) | Systems and methods for ranking data visualizations | |
| US9244887B2 (en) | Computer-implemented systems and methods for efficient structuring of time series data | |
| CN107368700A (en) | Based on the microbial diversity interaction analysis system and method for calculating cloud platform | |
| US9613102B2 (en) | Systems and methods for ranking data visualizations | |
| US20190034945A1 (en) | Information processing system, information processing method, and information processing program | |
| CN112860769B (en) | Energy planning data management system | |
| US8145635B1 (en) | Dimensional data explorer | |
| CN116384889A (en) | Intelligent analysis method for information big data based on natural language processing technology | |
| CN110321914A (en) | A kind of Oil Quality Analysis managing and control system | |
| CN116739836A (en) | Restaurant data analysis method and system based on knowledge graph | |
| CN116579804A (en) | Holiday commodity sales prediction method, holiday commodity sales prediction device and computer storage medium | |
| WO2017221856A1 (en) | Analysis device, analysis method, and recording medium | |
| CN118245316A (en) | Foreground data monitoring method and system of three-party data information platform | |
| CN114860737A (en) | Processing method, device, equipment and medium of teaching and research data | |
| CN112506930B (en) | Data insight system based on machine learning technology | |
| CN114511174A (en) | Service index map construction method and device | |
| JPH09282307A (en) | Commodity sales trend analysis method and system therefor | |
| CN111883204A (en) | IncRNA and mRNA correlation analysis system based on biological cloud platform | |
| CA3113501A1 (en) | Systems and methods for ranking data visualizations | |
| CN118331952B (en) | Financial data cleaning management system and method based on big data | |
| US7606846B2 (en) | Managing large OLAP in an analytical context | |
| Xanthidis et al. | Data Analytics and Data Visualization with Python | |
| Sodikov et al. | Analyzing the large amount of data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |