CN117076536A - Automated statistical information collection method, electronic device, and storage medium - Google Patents

Automated statistical information collection method, electronic device, and storage medium Download PDF

Info

Publication number
CN117076536A
CN117076536A CN202311037387.3A CN202311037387A CN117076536A CN 117076536 A CN117076536 A CN 117076536A CN 202311037387 A CN202311037387 A CN 202311037387A CN 117076536 A CN117076536 A CN 117076536A
Authority
CN
China
Prior art keywords
collection
data table
statistical information
target database
execution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311037387.3A
Other languages
Chinese (zh)
Inventor
翁俊生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Bank Co Ltd
Original Assignee
Ping An Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Bank Co Ltd filed Critical Ping An Bank Co Ltd
Priority to CN202311037387.3A priority Critical patent/CN117076536A/en
Publication of CN117076536A publication Critical patent/CN117076536A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2291—User-Defined Types; Storage management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Fuzzy Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application provides an automatic statistical information collection method, electronic equipment and a storage medium, wherein the automatic statistical information collection method comprises the following steps: the policy center determines a collection frequency based on the type of stock data table in the target database; the method comprises the steps that a policy center sends a data collection instruction to an execution engine, so that the execution engine collects the statistical information of a target database based on execution parameters and collection frequency and stores the statistical information of the target database, the statistical information of the target database is sent to an analysis engine, the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the execution result of the data collection instruction is fed back to the policy center; the policy center receives the analysis results sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis results. The application can automatically collect the statistical information of the database. Compared with the prior art, the application has the advantages of high efficiency, flexibility and accuracy.

Description

Automated statistical information collection method, electronic device, and storage medium
Technical Field
The application relates to the field of financial data processing, in particular to an automatic statistical information collection method, electronic equipment and a storage medium.
Background
The statistical information of the database is a type of information describing the table, index size, scale, data distribution status, etc. in the database. In general, the statistical information of the database is used to assist the CBO (query optimizer) to calculate the "costs" of various possible "execution plans", and further select the execution scheme with the lowest "cost", so that the presence or absence of the statistical information of the database and the accuracy will ultimately affect the CBO to make the optimal selection.
At present, the collection mode of statistical information of a database is mainly as follows: the manual collection is performed based on the API provided by the database, and the manual collection mode cannot realize automatic collection, so that the system has the defect of low collection efficiency. Further, the manual collection method is easy to miss part of information, so that the information is inaccurate. Further, the manual collection method cannot analyze the collected information and adjust the collection strategy according to the analysis result, so that the manual collection method has the defect of low flexibility.
Disclosure of Invention
An embodiment of the application aims to provide an automatic statistical information collection method, electronic equipment and a storage medium, which are used for automatically collecting statistical information of a database. Compared with the prior art, the application has the advantages of high efficiency, flexibility and accuracy.
In a first aspect, the present application provides an automated statistical information collection method applied to an automated statistical information collection framework, wherein the automated statistical information collection framework comprises a policy center, an execution engine, and an analysis engine, the method comprising:
the policy center determining a collection frequency based on the type of stock data table in the target database;
the policy center sends a data collection instruction to the execution engine so that the execution engine collects the statistical information of the target database and stores the statistical information of the target database based on the execution parameters and the collection frequency, and sends the statistical information of the target database to the analysis engine, so that the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the execution result of the data collection instruction is fed back to the policy center, wherein the data collection instruction carries the collection frequency;
the policy center receives an execution result of the data collection instruction sent by the execution engine;
the policy center receives the analysis result sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis result.
In the first aspect of the present application, the policy center may determine a collection frequency based on a type of an inventory data table in a target database, and may further send a data collection instruction to the execution engine, so that the execution engine may collect and save statistical information of the target database based on an execution parameter and the collection frequency, and send the statistical information of the target database to the analysis engine, so that the analysis engine may analyze the statistical information of the target database and obtain an analysis result. Further, the policy center can determine whether the statistics information collection of the database is successful or not by receiving the execution result of the data collection instruction sent by the execution engine, and finally, the statistics information of the database is automatically collected. And meanwhile, the policy center receives the analysis result sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis result.
Compared with the prior art, the application realizes automatic collection of statistical information without manual participation, so that the statistical information is not influenced by factors such as manual energy and the like, and has higher collection efficiency. Meanwhile, the application can reduce the occurrence probability of the event of information error caused by artificial omission and misoperation, and further can improve the accuracy of statistical information. In yet another aspect, the present application may enable greater flexibility by adjusting the collection frequency or initiating a re-collection instruction based on the analysis results.
In an alternative embodiment, the types of the stock data table include a normal table type, a hash table type and a partition table type;
and determining the collection frequency based on the type of the stock data table in the target database, comprising:
when the type of the stock data table is a common table type, the collection frequency is once a day;
when the type of the stock data table is a hash table type, the collection frequency is once a week;
when the type of the stock data table is a partition table type, the collection frequency is once a month.
This alternative embodiment may set a collection frequency once a day for a normal table, may set a collection frequency once a week for a hash table, and may set a collection frequency once a month for a partition table.
In an alternative embodiment, the method further comprises:
the execution engine identifies the operation stage of the bank core system based on the current system time;
when the operation stage of the bank core system is a business peak period, the execution engine determines a serial operation mode as the execution parameter;
and when the operation stage of the bank core system is a service peak period, the execution engine determines a parallel operation mode as the execution parameter.
In this optional embodiment, the execution engine may identify an operation stage of the bank core system based on the current system time, and further determine, when the operation stage of the bank core system is a peak business stage, a serial operation mode as the execution parameter, and determine, when the operation stage of the bank core system is a peak business stage, a parallel operation mode as the execution parameter, so that it is possible to avoid that the process of acquiring statistical information occupies too many hardware resources of the bank core system during the peak business stage, and fully utilize the hardware resources of the bank core system during the peak business stage.
In an alternative embodiment, the method further comprises:
and when the operation stage of the bank core system is a business stability stage, the execution engine backs up the statistical information of the target database.
In this optional embodiment, the execution engine can backup the statistics information of the target database during the service stabilization period, so as to ensure the reliability of information backup.
In an alternative embodiment, the statistics of the target database include a redox log;
and the analysis engine analyzes the statistical information of the target database and obtains an analysis result, comprising:
and judging whether a large-batch count operation aiming at the target database exists or not based on the redox log, and if the large-batch count operation aiming at the target database exists, taking the information of a data table of which the large-batch count operation occurs as the analysis result.
In this optional embodiment, the analysis engine determines, based on the redox log, whether there is a large-batch brush count operation for the target database, and if there is a large-batch brush count operation for the target database, uses information of a data table in which the large-batch brush count operation occurs as the analysis result.
In an alternative embodiment, the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and further includes:
judging whether a new service data table exists or not based on the redox log, and if the new service data table exists, taking the information of the new service data table as the analysis result.
In this optional embodiment, the analysis engine may determine, based on the redox log, whether a new service data table exists, and if the new service data table exists, use information of the new service data table as the analysis result.
In an alternative embodiment, the statistics of the target database further includes a service log;
and the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the method further comprises the following steps:
judging whether a data table with overtime inquiry exists or not based on the service log, and if the data table with overtime inquiry exists, taking the information of the data table with overtime inquiry as the analysis result.
In this optional embodiment, the analysis engine may determine, based on the service log, whether a data table with a timeout is present, and if the data table with a timeout is present, use information of the data table with a timeout as the analysis result.
In an alternative embodiment, the adjusting the collection frequency or initiating a re-collection instruction based on the analysis result includes:
when the analysis result carries information of the data table with the large-batch brushing operation, and the information of the data table with the large-batch brushing operation represents that the data modification quantity of the data table with the large-batch brushing operation is more than 20%, a re-collection instruction of the data table with the large-batch brushing operation is initiated;
when the analysis result carries the information of the newly added service data table, initiating a re-collection instruction aiming at the newly added service data table;
and when the analysis result carries the information of the data table with overtime query, modifying the collection frequency of the data table with overtime query.
In this optional embodiment, when the analysis result carries information of the data table in which the large-batch brush number operation occurs, and the information of the data table in which the large-batch brush number operation occurs characterizes that a data modification amount of the data table in which the large-batch brush number operation occurs is greater than 20%, the policy center can initiate a re-collection instruction for the data table in which the large-batch brush number operation occurs; and when the analysis result carries the information of the newly added service data table, initiating a re-collection instruction aiming at the newly added service data table, and meanwhile, when the analysis result carries the information of the data table with overtime query, the policy center can modify the collection frequency of the data table with overtime query.
In a second aspect, the present application provides an electronic device comprising:
a processor; and
a memory configured to store machine readable instructions that, when executed by the processor, perform the automated statistical information collection method of any of the preceding embodiments.
The electronic device of the second aspect of the present application can implement automatic collection of statistical information by executing an automatic statistical information collection method, and has advantages of high efficiency, high flexibility, and high accuracy compared with the prior art.
In a third aspect, the present application provides a storage medium storing a computer program for execution by a processor of the automated statistical information collection method according to any one of the preceding embodiments.
The storage medium of the third aspect of the present application can implement automatic collection of statistical information by executing an automated statistical information collection method, and has advantages of high efficiency, high flexibility, and high accuracy compared with the prior art.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an automated statistical information collection method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an automated statistics collection framework disclosed in an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.
Example 1
Referring to fig. 1, fig. 1 is a flow chart of an automated statistical information collection method according to an embodiment of the present application, wherein the automated statistical information collection method is applied to an automated statistical information collection framework. Further, referring to fig. 2, fig. 2 is a schematic diagram of an automated statistics collection framework according to an embodiment of the present application. As shown in fig. 2, the dynamic statistics information collection framework includes a policy center, an execution engine and an analysis engine, where the policy center, the execution engine and the analysis engine may be function modules deployed on the same server, and may be function modules deployed on different entity devices respectively. Further, the policy center is communicatively coupled to the enforcement engine and the analysis engine, respectively, and the enforcement engine is communicatively coupled to the analysis engine. Further, the automated statistics information collection framework is in communication connection with the bank core system, so that a database of the bank core system can be accessed, wherein the database of the bank core system can be used as a target database, and an application service of the bank core system can generate data and record the data in the target database, so that a stock data table of the target database is formed.
In an embodiment of the present application, as shown in fig. 1, the method of the embodiment of the present application includes the following steps:
101. the policy center determines a collection frequency based on the type of stock data table in the target database;
102. the method comprises the steps that a policy center sends a data collection instruction to an execution engine, so that the execution engine collects the statistical information of a target database based on execution parameters and collection frequency and stores the statistical information of the target database, the statistical information of the target database is sent to an analysis engine, the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the execution result of the data collection instruction is fed back to the policy center, wherein the data collection instruction carries the collection frequency;
103. the policy center receives an execution result of a data collection instruction sent by an execution engine;
104. the policy center receives the analysis results sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis results.
As can be seen from fig. 1, in the embodiment of the present application, the policy center can determine the collection frequency based on the type of the stock data table in the target database, and further can send a data collection instruction to the execution engine, so that the execution engine can collect the statistics of the target database and store the statistics of the target database based on the execution parameters and the collection frequency, and send the statistics of the target database to the analysis engine, so that the analysis engine can analyze the statistics of the target database and obtain the analysis result. Further, the policy center can determine whether the statistics information collection of the database is successful or not by receiving the execution result of the data collection instruction sent by the execution engine, and finally, the statistics information of the database is automatically collected. Meanwhile, the policy center receives the analysis result sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis result.
Compared with the prior art, the embodiment of the application realizes automatic collection of statistical information without manual participation, so that the statistical information is not influenced by factors such as manual energy and the like, and the collection efficiency is higher. Meanwhile, the embodiment of the application can reduce the occurrence probability of the event of information error caused by artificial omission and misoperation, and further can improve the accuracy of statistical information. In still another aspect, the embodiment of the present application can adjust the collection frequency or initiate the re-collection instruction based on the analysis result, thereby having higher flexibility.
In an embodiment of the present application, for step 101, the stock data table in the target database may include a plurality of data tables, for example, may include 10 data tables. Further, the type of each stock data table may be different, for example, there are 5 normal tables, 3 hash tables, and 2 partition tables among 10 data tables.
In the embodiment of the present application, for step 103, the execution result of the data collection instruction sent by the execution engine is received to determine whether the execution engine has executed the data collection instruction issued by the execution engine, where if the execution engine has executed the data collection instruction, "1" is returned to the policy center as the execution result, so that the policy center determines that the execution engine is already executing the data collection instruction.
In the embodiment of the present application, as an optional implementation manner, the types of the stock data table include a normal table type, a hash table type and a partition table type, and accordingly, the steps include: determining a collection frequency based on the type of stock data table in the target database, comprising the sub-steps of:
when the type of the stock data table is the common table type, the collection frequency is once a day;
when the type of the stock data table is a hash table type, the collection frequency is once a week;
when the type of the stock data table is the partition table type, the collection frequency is once a month.
This alternative embodiment may set a collection frequency once a day for a normal table, may set a collection frequency once a week for a hash table, and may set a collection frequency once a month for a partition table.
For the above alternative embodiment, since the data size of the common table is small and the update speed is high, the data needs to be collected once a day, so that the latest statistical information can be ensured to be obtained. The partition table is updated only by data which is large in data amount and is usually accumulated for one month, so that the partition table can be collected once a month.
For the alternative embodiment described above, the data in the common table is stored in rows, each row of data being stored consecutively in the table. And a hash table refers to a data structure in which data is mapped into a fixed-size table by a hash function. The partition table refers to a table type for dividing data into a plurality of parts, wherein each part is called a partition, the partition table can store the data in different partitions in a scattered manner, and each partition can perform operations of inquiring, inserting, deleting and the like independently.
In an embodiment of the present application, as an optional implementation manner, the method of the embodiment of the present application further includes:
the execution engine identifies the operation stage of the bank core system based on the current system time;
when the operation stage of the bank core system is a business peak period, the execution engine determines a serial operation mode as an execution parameter;
when the operation phase of the bank core system is the service peak period, the execution engine determines the parallel operation mode as the execution parameter.
In this optional embodiment, the execution engine can identify the operation stage of the bank core system based on the current system time, and further when the operation stage of the bank core system is a service peak period, the execution engine determines the serial operation mode as an execution parameter, and when the operation stage of the bank core system is a service peak period, the execution engine determines the parallel operation mode as an execution parameter, so that the process of acquiring statistical information can be prevented from occupying excessive hardware resources of the bank core system during the service peak period, and the hardware resources of the bank core system can be fully utilized during the service peak period.
For the above alternative embodiment, one specific way for the execution engine to identify the operational phase of the banking core system based on the current system time may be:
if the current system time is the time point between 22 points and 05 points of the next day, determining that the operation phase of the bank core system is the service stability phase;
if the current system time is a time point between 05 points and 9 points, determining the operation phase of the bank core system as a business flat beacon period;
if the current system time is the time point between 9 and 22 points, determining the operation stage of the bank core system as a service high beacon period.
In an embodiment of the present application, as an optional implementation manner, the method of the embodiment of the present application further includes the following steps:
when the operation stage of the bank core system is the business stability stage, the execution engine backs up the statistical information of the target database.
In this optional embodiment, the execution engine can backup the statistics information of the target database during the service stabilization period, so as to ensure the reliability of information backup.
For the above optional implementation manner, since the system in the service stability period is relatively stable, the terminal of data backup is not easy to occur, so that the complete backup of data can be ensured, and the reliability of the statistics backup can be ensured. Further, the execution engine may backup statistics of the target database based on the distributed storage system.
In an embodiment of the present application, as an optional implementation manner, the statistical information of the target database includes a redox log;
and the analysis engine analyzes the statistical information of the target database and obtains an analysis result, comprising:
judging whether a large-batch count operation aiming at the target database exists or not based on the redox log, and if the large-batch count operation aiming at the target database exists, taking the information of a data table of which the large-batch count operation occurs as an analysis result.
In this alternative embodiment, the analysis engine determines, based on the redox log, whether there is a large-batch brush count operation for the target database, and if there is a large-batch brush count operation for the target database, the information of the data table in which the large-batch brush count operation occurs is taken as an analysis result.
For the above alternative embodiment, the target database may be an Oracle database, and the redox log may refer to a redox log of the Oracle database, where the redox log is a physical log used to record modification operations of the database. In a database system, when a transaction performs a modification operation, it records the modification operation in a redox log, and then applies the modification operation to the database. If a failure or exception occurs, the database system may restore the data to a consistent state by replaying the redox log. The redox log typically employs a WAL (Write-Ahead-Logging) mechanism, and is therefore also referred to as a WAL log. Unlike the undo log, the redo log records modification operations to the database, while the undo log records the foreground and background of the operations for rollback operations.
In an embodiment of the present application, as an optional implementation manner, the steps include: the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the method further comprises the following substeps:
judging whether a new service data table exists or not based on the redox log, and if the new service data table exists, taking the information of the new service data table as an analysis result.
In this alternative embodiment, the analysis engine can determine whether a new service data table exists based on the redox log, and if the new service data table exists, take information of the new service data table as an analysis result.
In an alternative embodiment, the statistics of the target database further includes a service log, wherein the service log records the process and result of the service operation, including modification and query operations to the database, and the like. Accordingly, the steps of: the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the method further comprises the following substeps:
judging whether a data table with overtime inquiry exists or not based on the service log, and if the data table with overtime inquiry exists, taking the information of the data table with overtime inquiry as an analysis result.
In this alternative embodiment, the analysis engine can determine whether there is a data table with a timeout in the query based on the service log, and if there is a data table with a timeout in the query, the information of the data table with a timeout in the query is used as the analysis result.
In an embodiment of the present application, as an optional implementation manner, the steps include: adjusting the collection frequency or initiating a re-collection instruction based on the analysis result, comprising the steps of:
when the analysis result carries information of a data table with a large-batch count operation, and the information of the data table with the large-batch count operation represents that the data modification quantity of the data table with the large-batch count operation is more than 20%, a re-collection instruction for the data table with the large-batch count operation is initiated;
when the analysis result carries the information of the newly added service data table, initiating a re-collection instruction aiming at the newly added service data table;
when the analysis result carries information of the data table with overtime inquiry, the collection frequency of the data table with overtime inquiry is modified.
In the optional implementation manner, the policy center can initiate a re-collection instruction for the data table with the large-batch brush number operation when the analysis result carries information of the data table with the large-batch brush number operation and the information of the data table with the large-batch brush number operation represents that the data modification quantity of the data table with the large-batch brush number operation is more than 20%; and when the analysis result carries information of the newly added service data table, a re-collection instruction aiming at the newly added service data table is initiated, and meanwhile, the strategy center can modify the collection frequency of the data table with overtime query when the analysis result carries information of the data table with overtime query.
Example III
Referring to fig. 3, fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 3, the electronic device according to the embodiment of the present application includes:
a processor 201; and
a memory 202 configured to store machine readable instructions that, when executed by a processor, perform the automated statistical information collection method of any of the foregoing embodiments.
The electronic equipment of the embodiment of the application can enable the policy center to determine the collection frequency based on the type of the stock data table in the target database by executing the automatic statistical information collection method, and further can send the data collection instruction to the execution engine so that the execution engine can collect the statistical information of the target database and store the statistical information of the target database based on the execution parameters and the collection frequency, and send the statistical information of the target database to the analysis engine so that the analysis engine can analyze the statistical information of the target database and obtain an analysis result. Further, the policy center can determine whether the statistics information collection of the database is successful or not by receiving the execution result of the data collection instruction sent by the execution engine, and finally, the statistics information of the database is automatically collected. Meanwhile, the policy center receives the analysis result sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis result.
Compared with the prior art, the embodiment of the application realizes automatic collection of statistical information without manual participation, so that the statistical information is not influenced by factors such as manual energy and the like, and the collection efficiency is higher. Meanwhile, the embodiment of the application can reduce the occurrence probability of the event of information error caused by artificial omission and misoperation, and further can improve the accuracy of statistical information. In still another aspect, the embodiment of the present application can adjust the collection frequency or initiate the re-collection instruction based on the analysis result, thereby having higher flexibility.
Example IV
An embodiment of the present application provides a storage medium storing a computer program that is executed by a processor to perform the automated statistical information collection method according to any one of the foregoing embodiments.
The storage medium of the embodiment of the application can enable the policy center to determine the collection frequency based on the type of the stock data table in the target database by executing the automatic statistical information collection method, and further can send the data collection instruction to the execution engine so that the execution engine can collect the statistical information of the target database and store the statistical information of the target database based on the execution parameter and the collection frequency, and send the statistical information of the target database to the analysis engine so that the analysis engine can analyze the statistical information of the target database and obtain an analysis result. Further, the policy center can determine whether the statistics information collection of the database is successful or not by receiving the execution result of the data collection instruction sent by the execution engine, and finally, the statistics information of the database is automatically collected. Meanwhile, the policy center receives the analysis result sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis result.
Compared with the prior art, the embodiment of the application realizes automatic collection of statistical information without manual participation, so that the statistical information is not influenced by factors such as manual energy and the like, and the collection efficiency is higher. Meanwhile, the embodiment of the application can reduce the occurrence probability of the event of information error caused by artificial omission and misoperation, and further can improve the accuracy of statistical information. In still another aspect, the embodiment of the present application can adjust the collection frequency or initiate the re-collection instruction based on the analysis result, thereby having higher flexibility.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
Further, the units described as separate units may or may not be physically separate, and units displayed as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Furthermore, functional modules in various embodiments of the present application may be integrated together to form a single portion, or each module may exist alone, or two or more modules may be integrated to form a single portion.
It should be noted that the functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM) random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above embodiments of the present application are only examples, and are not intended to limit the scope of the present application, and various modifications and variations will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (10)

1. An automated statistical information collection method, the method being applied to an automated statistical information collection framework, wherein the automated statistical information collection framework comprises a policy center, an enforcement engine, and an analysis engine, the method comprising:
the policy center determining a collection frequency based on the type of stock data table in the target database;
the policy center sends a data collection instruction to the execution engine so that the execution engine collects the statistical information of the target database and stores the statistical information of the target database based on the execution parameters and the collection frequency, and sends the statistical information of the target database to the analysis engine, so that the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the execution result of the data collection instruction is fed back to the policy center, wherein the data collection instruction carries the collection frequency;
the policy center receives an execution result of the data collection instruction sent by the execution engine;
the policy center receives the analysis result sent by the analysis engine and adjusts the collection frequency or initiates a re-collection instruction based on the analysis result.
2. The method of claim 1, wherein the types of inventory data tables include a normal table type, a hash table type, and a partition table type;
and determining the collection frequency based on the type of the stock data table in the target database, comprising:
when the type of the stock data table is a common table type, the collection frequency is once a day;
when the type of the stock data table is a hash table type, the collection frequency is once a week;
when the type of the stock data table is a partition table type, the collection frequency is once a month.
3. The method of claim 1, wherein the method further comprises:
the execution engine identifies the operation stage of the bank core system based on the current system time;
when the operation stage of the bank core system is a business peak period, the execution engine determines a serial operation mode as the execution parameter;
and when the operation stage of the bank core system is a service peak period, the execution engine determines a parallel operation mode as the execution parameter.
4. A method as claimed in claim 3, wherein the method further comprises:
and when the operation stage of the bank core system is a business stability stage, the execution engine backs up the statistical information of the target database.
5. The method of claim 1, wherein the statistics of the target database comprise a redox log;
and the analysis engine analyzes the statistical information of the target database and obtains an analysis result, comprising:
and judging whether a large-batch count operation aiming at the target database exists or not based on the redox log, and if the large-batch count operation aiming at the target database exists, taking the information of a data table of which the large-batch count operation occurs as the analysis result.
6. The method of claim 5, wherein the analysis engine analyzes the statistics of the target database and obtains analysis results, further comprising:
judging whether a new service data table exists or not based on the redox log, and if the new service data table exists, taking the information of the new service data table as the analysis result.
7. The method of claim 6, wherein the statistics of the target database further comprise a traffic log;
and the analysis engine analyzes the statistical information of the target database and obtains an analysis result, and the method further comprises the following steps:
judging whether a data table with overtime inquiry exists or not based on the service log, and if the data table with overtime inquiry exists, taking the information of the data table with overtime inquiry as the analysis result.
8. The method of claim 7, wherein the adjusting the collection frequency or initiating a re-collection instruction based on the analysis result comprises:
when the analysis result carries information of the data table with the large-batch brushing operation, and the information of the data table with the large-batch brushing operation represents that the data modification quantity of the data table with the large-batch brushing operation is more than 20%, a re-collection instruction of the data table with the large-batch brushing operation is initiated;
when the analysis result carries the information of the newly added service data table, initiating a re-collection instruction aiming at the newly added service data table;
and when the analysis result carries the information of the data table with overtime query, modifying the collection frequency of the data table with overtime query.
9. An electronic device, comprising:
a processor; and
a memory configured to store machine readable instructions that, when executed by the processor, perform the automated statistical information collection method of any one of claims 1-8.
10. A storage medium storing a computer program for execution by a processor of the automated statistical information collection method according to any one of claims 1-8.
CN202311037387.3A 2023-08-16 2023-08-16 Automated statistical information collection method, electronic device, and storage medium Pending CN117076536A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311037387.3A CN117076536A (en) 2023-08-16 2023-08-16 Automated statistical information collection method, electronic device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311037387.3A CN117076536A (en) 2023-08-16 2023-08-16 Automated statistical information collection method, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
CN117076536A true CN117076536A (en) 2023-11-17

Family

ID=88718888

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311037387.3A Pending CN117076536A (en) 2023-08-16 2023-08-16 Automated statistical information collection method, electronic device, and storage medium

Country Status (1)

Country Link
CN (1) CN117076536A (en)

Similar Documents

Publication Publication Date Title
CN107563887B (en) AS 400-based zero-halt daily cutting method for bank core accounting system
US6816860B2 (en) Database load distribution processing method and recording medium storing a database load distribution processing program
US9672244B2 (en) Efficient undo-processing during data redistribution
CN1794181A (en) Finally agent optimization method and system of source in reorder two-stage refering
CN105989163A (en) Data real-time processing method and system
CN116700634B (en) Garbage recycling method and device for distributed storage system and distributed storage system
CN1871586A (en) Tracking space usage in a database
CN117076536A (en) Automated statistical information collection method, electronic device, and storage medium
CN111382028B (en) Method, device and server for processing date switching errors of batch processing system
CN117472652A (en) Data backup method, device and system of cloud computing operation and maintenance platform
CN111404737A (en) Disaster recovery processing method and related device
CN115408342A (en) File processing method and device and electronic equipment
CN111813833B (en) Real-time two-degree communication relation data mining method
CN113869920A (en) Business opportunity stage division method, system, computer equipment and storage medium
CN111901448B (en) CTDB virtual IP balance distribution method for cluster node fault scene
CN111459946B (en) Data table rapid summarizing method and device, computer equipment and storage medium
CN113360551A (en) Method and system for storing and rapidly counting time sequence data in shooting range
CN108874325B (en) Data printing method and system
CN117992257B (en) Parallel data acquisition and processing method for distributed database
CN113535469B (en) Switching method and switching system for disaster recovery database
CN114003622B (en) Huge transaction increment synchronization method between transaction type databases
CN115203288A (en) Database information statistical method and device
CN116975124A (en) Method, device, equipment and storage medium for analyzing slow log of distributed database
CN112347098B (en) Database table splitting method, system, electronic equipment and storage medium
CN118057264A (en) Automatic charging and discharging method and device for server battery

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination