CN113535726A - Database capacity expansion method and device - Google Patents

Database capacity expansion method and device Download PDF

Info

Publication number
CN113535726A
CN113535726A CN202110803627.0A CN202110803627A CN113535726A CN 113535726 A CN113535726 A CN 113535726A CN 202110803627 A CN202110803627 A CN 202110803627A CN 113535726 A CN113535726 A CN 113535726A
Authority
CN
China
Prior art keywords
data volume
data
database
storage
historical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110803627.0A
Other languages
Chinese (zh)
Inventor
张明永
郁志清
何彬
高天宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202110803627.0A priority Critical patent/CN113535726A/en
Publication of CN113535726A publication Critical patent/CN113535726A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/285Clustering or classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present disclosure provides a database capacity expansion method, which can be applied to the financial field and the database technical field. The database capacity expansion method comprises the following steps: acquiring historical storage information of a database in an interface calling mode; obtaining the estimated growth data volume of the database in a preset management period according to the historical storage information; generating a capacity expansion script according to the estimated increase data volume; and expanding the capacity of the database by using the capacity expansion script. The present disclosure also provides a database capacity expansion device, an apparatus, a storage medium, and a program product.

Description

Database capacity expansion method and device
Technical Field
The present disclosure relates to the field of finance and database technologies, and more particularly, to a method, an apparatus, a device, a medium, and a program product for expanding a database.
Background
Oracle data space management is one of the most fundamental, important jobs for database maintenance.
The inventor finds that, in the process of implementing the concept disclosed herein, the maintenance method of the database in the related art is mainly based on manually generating a capacity expansion script, and has the following main problems:
the capacity expansion requirement mainly depends on monitoring alarm and operation and maintenance experience, and the initiative and the normalization are not enough.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a database capacity expansion method, apparatus, device, medium, and program product.
According to a first aspect of the present disclosure, there is provided a database capacity expansion method, including:
acquiring historical storage information of a database in an interface calling mode;
obtaining the estimated growth data volume of the database in a preset management period according to the historical storage information;
generating a capacity expansion script according to the estimated increase data volume;
and expanding the capacity of the database by using the capacity expansion script.
According to an embodiment of the present disclosure, the history storage information includes a first data amount of a first class storage space and a first history data amount, where the first data amount includes a data amount of the first class storage space at a first time point to be expanded, and the first history data amount includes a data amount of the first class storage space at a second time point having a first time span with the first time point to be expanded, where the first time span is equal to the preset management period;
the obtaining of the estimated growth data volume of the database in a preset management period according to the historical storage information includes:
acquiring a first storage threshold value of the first category storage space;
and obtaining the estimated increase data volume of the first class storage space in a preset management period according to the first storage threshold, the first data volume and the first historical data volume.
According to an embodiment of the present disclosure, the obtaining an estimated increase data volume of the first class storage space in a preset management period according to the first storage threshold, the first data volume, and the first historical data volume includes:
generating a first difference value according to the first data volume and the first historical data volume;
and obtaining the estimated increase data volume of the first class storage space in a preset management period according to the first difference and the first storage threshold.
According to an embodiment of the present disclosure, after obtaining the estimated growth data amount of the first class storage space in the preset management period, the method further includes:
acquiring a first rated capacity of the first category of storage space;
generating a maximum safe storage amount according to the first rated capacity and the first storage threshold value;
generating a total estimated data amount according to the first data amount and the estimated growth data amount;
and under the condition that the estimated data total amount is less than or equal to the maximum safe storage amount, generating a capacity expansion script according to the estimated increase data amount.
According to an embodiment of the present disclosure, the history storage information includes a first total amount of history data of a second type of storage space, where the first total amount of history data includes a total amount of data of the second type of storage space in a history management period having a second time span with the preset management period; the second type of storage space has an initial storage capacity;
the obtaining of the estimated growth data volume of the database in a preset management period according to the historical storage information includes:
generating a second difference value according to the total amount of the first historical data and the initial storage capacity;
and taking the second difference value as the estimated growth data amount.
According to an embodiment of the present disclosure, the history storage information includes a second data volume and a second total history data volume of a third category of storage space, where the second data volume includes a data volume of the third category of storage space at a second time point to be expanded, and the second total history data volume includes a data volume of the third category of storage space in a history management period having a third time span with the second time point to be expanded;
the obtaining of the estimated growth data volume of the database in a preset management period according to the historical storage information includes:
generating an average data volume according to the second historical data total and the third time span;
generating a standard data capacity of the third type storage space according to the average data volume and a preset time threshold of the third type storage space;
and obtaining the estimated growth data volume according to the standard data volume and the second rated data volume of the third type of storage space.
According to an embodiment of the present disclosure, the generating a capacity expansion script according to the estimated growth data amount includes:
acquiring N data files according to the estimated growth data volume, wherein the sum of the data volumes of the N data files is greater than or equal to the estimated growth data volume, and N is an integer;
acquiring identification information of each data file in the N data files;
and generating the capacity expansion script by taking the identification information as a script parameter.
A second aspect of the present disclosure provides a database capacity expansion apparatus, including:
the acquisition module is used for acquiring historical storage information of the database in an interface calling mode;
the estimation module is used for obtaining the estimated increase data volume of the database in a preset management period according to the historical storage information;
the generating module is used for generating a capacity expansion script according to the estimated growth data volume; and
and the capacity expansion module is used for expanding the capacity of the database by using the capacity expansion script.
A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the database capacity expansion method.
A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions, which when executed by a processor, cause the processor to execute the above database capacity expansion method.
A fifth aspect of the present disclosure also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the database capacity expansion method.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:
fig. 1 is an application scenario diagram schematically illustrating an application scenario diagram of a database capacity expansion method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;
FIG. 2 is a flow chart that schematically illustrates a method for database capacity expansion, in accordance with an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart for obtaining an estimated growth data volume of a database in a preset management period according to historical storage information according to an embodiment of the present disclosure;
FIG. 4 is a flowchart schematically illustrating obtaining an estimated increase data volume of a first class of storage space within a preset management period according to a first storage threshold, a first data volume, and a first historical data volume according to an embodiment of the present disclosure;
FIG. 5 schematically illustrates a flow diagram of a method of database capacity expansion according to another embodiment of the present disclosure;
FIG. 6 is a flow chart schematically illustrating obtaining an estimated growth data volume of a database in a preset management period according to historical storage information according to another embodiment of the present disclosure;
FIG. 7 is a flow chart schematically illustrating obtaining an estimated growth data volume of a database in a preset management period according to historical storage information according to another embodiment of the present disclosure;
FIG. 8 is a flow diagram that schematically illustrates generation of a capacity expansion script based on an estimated growth data volume, in accordance with an embodiment of the present disclosure;
FIG. 9 is a block diagram illustrating an exemplary configuration of a database capacity expansion device according to an embodiment of the disclosure; and
fig. 10 schematically illustrates a block diagram of an electronic device suitable for implementing a database capacity expansion method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
In the process of implementing the concept of the present disclosure, the inventors found that, in the related art, the operation and maintenance of the database is mainly based on manually generating a capacity-expansion script, and the following problems mainly exist:
1. the capacity expansion requirement mainly depends on monitoring alarm and operation and maintenance experience, and the initiative and the normalization are not enough.
2. When the capacity of the database needs to be expanded, the capacity expansion task is usually manually distributed to corresponding operation and maintenance personnel, and then the capacity expansion script is written by the corresponding operation and maintenance personnel.
In order to solve the problems in the related art, the present disclosure provides a database capacity expansion method, which can be applied to the financial field and the database technology field. The database capacity expansion method comprises the following steps: acquiring historical storage information of a database in an interface calling mode; obtaining the estimated growth data volume of the database in a preset management period according to the historical storage information; generating a capacity expansion script according to the estimated increase data volume; and expanding the capacity of the database by using the capacity expansion script. The present disclosure also provides a database capacity expansion device, an apparatus, a storage medium, and a program product.
It should be noted that the method and apparatus determined by the embodiment of the present disclosure may be used in the financial field and the database technology field, and may also be used in any field other than the financial field and the database technology field.
Fig. 1 is an application scenario diagram schematically illustrating an application scenario diagram of a database capacity expansion method, apparatus, device, medium, and program product according to an embodiment of the present disclosure.
As shown in fig. 1, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the database capacity expansion method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the database capacity expansion device provided by the embodiment of the present disclosure may be generally disposed in the server 105. The database capacity expansion method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, and 103 and/or the server 105. Accordingly, the database capacity expansion apparatus provided in the embodiment of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The following describes in detail a database capacity expansion method according to the disclosed embodiment with reference to fig. 2 to 8 based on the scenario described in fig. 1.
Fig. 2 schematically shows a flowchart of a database capacity expansion method according to an embodiment of the present disclosure.
As shown in fig. 2, the database capacity expansion method of this embodiment includes operations S201 to S204.
In operation S201, history storage information of the database is obtained through an interface call mode.
According to an embodiment of the present disclosure, the historical storage information may include storage information that can characterize the storage of the database over a certain period of time.
According to an embodiment of the present disclosure, the history storage information may include storage information of various kinds of storage spaces.
According to the embodiment of the present disclosure, the history storage information may be stored in a data file of the database in the form of a database table, but is not limited thereto, and the history storage information may also be stored in a data file of another database different from the database in the form of a database table.
According to the embodiments of the present disclosure, the history storage information may be stored in a certain database, but is not limited thereto, and the history storage information may also be stored in a plurality of different databases.
In operation S202, an estimated growth data amount of the database in a preset management period is obtained according to the history storage information.
According to the embodiment of the disclosure, through analyzing and analyzing the historical storage information, the estimated increase data volume of one or more types of storage spaces in the database in a preset management period can be estimated according to the storage condition of the storage spaces in the historical storage information in a certain period.
According to an embodiment of the present disclosure, the preset management period may include one month in the future, one quarter in the future, one day in the future, and the like, and the preset management period is not specifically limited in the embodiment of the present disclosure.
In operation S203, a capacity expansion script is generated according to the estimated growth data amount.
In operation S204, the database is expanded using the expansion script.
According to the embodiment of the disclosure, after the estimated growth data volume of the database in the preset management period is obtained according to the history storage information, the capacity expansion script may be automatically generated according to the estimated growth data volume.
According to the embodiment of the disclosure, as the user operates the application program or registers a new user, the amount of data in the database increases, however, the amount of data that can be stored in the database is limited, and when the amount of data stored in the database is greater than a preset threshold, the available space of the database generally needs to be expanded.
In the related art, a storage threshold is usually set for the database, and when the amount of data stored in the database is greater than the storage threshold, the database alarms to remind operation and maintenance managers to expand the database. After receiving the alarm information of the database, the operation and maintenance manager can formulate a capacity expansion demand according to prior experience, then distribute the capacity expansion demand to corresponding operation and maintenance personnel, and write a capacity expansion script according to the received capacity expansion demand and utilize the script to expand the database.
However, in the related art, the capacity expansion is performed only by relying on the alarm information, and the timeliness and the accuracy of the capacity expansion depend on the database capacity expansion mode of the operation and maintenance manager are poor.
In the embodiment of the disclosure, through analyzing and processing the historical storage information of the database, the estimated growth data volume of the database in the preset management period can be obtained, and a capacity expansion script can be automatically generated according to the estimated growth data volume to expand the database. The database capacity expansion method provided by the embodiment of the disclosure solves the technical problems of poor timeliness and accuracy in the related technology, realizes the technical effects of accurately and timely expanding the capacity of the database and reducing the alarm frequency of the database,
according to the embodiment of the disclosure, the history storage information includes a first data volume and a first history data volume of the first class storage space, where the first data volume includes a data volume of the first class storage space at a first time point to be expanded, and the first history data volume includes a data volume of the first class storage space at a second time point having a first time span with the first time point to be expanded, where the first time span is equal to the preset management period.
According to an embodiment of the present disclosure, the first category storage space may include, for example, a normal table space.
In the embodiment of the present disclosure, the preset management period may be, for example, 30 days, and thus, the first time span may also be 30 days.
According to the embodiment of the disclosure, the first time point to be expanded may be, for example, 6 months and 30 days, so that the first data amount may include, for example, the data amount of the database at 6 months and 30 days, and the first historical data amount may include, for example, the data amount of the database at 5 months and 30 days.
Fig. 3 schematically shows a flowchart for obtaining the estimated increase data volume of the database in the preset management period according to the history storage information according to the embodiment of the disclosure.
As shown in fig. 3, obtaining the estimated increase data amount of the database in the preset management period according to the history storage information in this embodiment includes operations S301 to S302.
In operation S301, a first storage threshold of a first category of storage space is obtained.
According to an embodiment of the present disclosure, the first storage threshold may comprise, for example, 75%, 80%, or 90% of a nominal storage data amount of the first category of storage space, or the like.
According to the embodiment of the present disclosure, the first storage threshold may be, for example, an alarm threshold of the first class storage space, that is, the first class storage space will alarm when the amount of data stored in the first class storage space exceeds the alarm threshold, but is not limited thereto, and the first storage threshold may also be lower than the alarm threshold of the first class storage space.
In operation S302, an estimated increase data volume of the first class storage space in a preset management period is obtained according to the first storage threshold, the first data volume, and the first historical data volume.
Fig. 4 schematically shows a flowchart for obtaining the estimated increase data volume of the first class storage space in the preset management period according to the first storage threshold, the first data volume and the first historical data volume according to the embodiment of the disclosure.
As shown in fig. 4, the obtaining of the estimated increase data volume of the first class storage space in the preset management period according to the first storage threshold, the first data volume, and the first historical data volume in this embodiment includes operations S401 to S402.
In operation S401, a first difference value is generated according to the first data amount and the first history data amount.
In operation S402, an estimated increase data amount of the first class storage space in a preset management period is obtained according to the first difference and the first storage threshold.
According to an embodiment of the present disclosure, the above-described operations S401 to S402 may be represented by the following formula (1).
Figure BDA0003164073300000101
Wherein, Y1Can represent the predicted growth data volume, x1May represent a first amount of data, x2May represent a first amount of historical data, f1A first storage threshold may be represented.
According to the embodiment of the disclosure, the data increment of the first category storage space in the first time span can be obtained by generating the first difference value according to the first data volume and the first historical data volume.
According to the embodiment of the disclosure, by generating the first ratio according to the first difference, that is, the data increment of the first class storage space in the first time span and the first storage threshold, the buffer storage space can be reserved for the first class storage space.
Fig. 5 schematically illustrates a flowchart of a database capacity expansion method according to another embodiment of the present disclosure.
As shown in fig. 5, the database capacity expansion method of this embodiment includes operations S501 to S504.
In operation S501, a first rated capacity of a first category of storage space is obtained.
According to an embodiment of the present disclosure, the first rated capacity may be a maximum capacity of the first category of storage space that is scalable.
In operation S502, a maximum safe storage amount is generated according to the first rated capacity and the first storage threshold.
According to the embodiment of the disclosure, the maximum safe storage amount may be an alarm threshold value of the first class storage space, that is, the database may alarm after the data amount in the first class storage space exceeds the maximum safe storage amount.
According to the embodiment of the disclosure, the maximum safe storage amount can be obtained by multiplying the first rated capacity by the first storage threshold.
In operation S503, a predicted total data amount is generated based on the first data amount and the predicted increase data amount.
According to an embodiment of the disclosure, the estimated total amount of data may be obtained by adding the first amount of data and the estimated increase amount of data.
According to an embodiment of the present disclosure, the total amount of predicted data may be the total amount of predicted data in the first class storage space after the first class storage space is expanded according to the predicted increase data amount.
In operation S504, under the condition that the estimated total data amount is less than or equal to the maximum safe storage amount, a capacity expansion script is generated according to the estimated increased data amount.
According to the embodiment of the disclosure, the fact that the estimated total data amount is less than or equal to the maximum safe storage amount indicates that after the first-class storage space is subjected to capacity expansion according to the estimated increase data amount, the total data amount stored in the first-class storage space does not exceed the alarm threshold, and the database does not give an alarm, so that the alarm frequency of the database can be reduced.
According to the embodiment of the disclosure, under the condition that the estimated total data amount is larger than the maximum safe storage amount, the total capacity of the database is indicated to be incapable of meeting the capacity expansion requirement, so that the storage gap information can be automatically generated and sent to the database maintenance personnel so as to upgrade the database.
According to the embodiment of the present disclosure, the storage gap information may include identification information of the first class storage space, a current data size of the first class storage space, a difference between a rated capacity of the first class storage space and a capacity expansion demand capacity, and the like.
According to the embodiment of the disclosure, the history storage information comprises a first history data total amount of the second category storage space, wherein the first history data total amount comprises a data total amount of the second category storage space in a history management period having a second time span with a preset management period; the second category of storage space has an initial storage capacity.
According to an embodiment of the present disclosure, the second category of storage space may include, for example, a time sequence table space.
According to an embodiment of the present disclosure, the time-series tablespace may be, for example, a tablespace created at a preset time interval, for example, a tablespace created at the beginning of the month of each month, or a tablespace created at the beginning of the quarter of each quarter.
According to an embodiment of the present disclosure, the second time span may comprise, for example, twelve months.
According to an embodiment of the present disclosure, the preset management period may be, for example, 7/month 1 to 7/month 30 in 2021, so that the history management period may be 7/month 1 to 7/month 30 in 2020.
Fig. 6 schematically shows a flowchart for obtaining the estimated increase data volume of the database in the preset management period according to the history storage information according to another embodiment of the present disclosure.
As shown in fig. 6, obtaining the estimated increase data amount of the database in the preset management period according to the history storage information in this embodiment includes operations S601 to S602.
In operation S601, a second difference value is generated according to the first total amount of history data and the initial storage capacity.
In operation S602, the second difference is used as the predicted increase data amount.
According to the embodiment of the disclosure, when the demand analysis is performed on the second type of storage space, a smaller initial storage capacity is often allocated to the second type of storage space, so as to meet the basic storage demand of the second type of storage space.
According to the embodiment of the disclosure, since the data storage amount of the database in each year has a certain regularity, for example, a certain month in a year has a shopping festival or a traditional festival, the data volume of the month generally has a certain increase, and thus, the data increase of the corresponding month in the year can be estimated according to the data volume of the same month in the previous year.
According to the embodiment of the disclosure, the history storage information includes a second data volume and a second history data total volume of the third category storage space, where the second data volume includes a data volume of the third category storage space at the second time point to be expanded, and the second history data total volume includes a data total volume of the third category storage space in a history management period having a third time span with the second time point to be expanded.
According to an embodiment of the present disclosure, the third category of storage space may include, for example, a log table space.
According to an embodiment of the present disclosure, the third time span may be, for example, 30 days.
According to an embodiment of the present disclosure, the second time point to be expanded may be, for example, 6 months and 28 days monday, and thus, the history management period having a third time span with the second time point to be expanded may be 5 months and 28 to 6 months and 28.
Fig. 7 schematically shows a flowchart for obtaining the estimated increase data volume of the database in the preset management period according to the history storage information according to another embodiment of the present disclosure.
As shown in fig. 7, obtaining the estimated increase data amount of the database in the preset management period according to the history storage information in this embodiment includes operations S701 to S703.
In operation S701, an average data amount is generated according to the second total historical data amount and the third time span.
According to an embodiment of the present disclosure, the average data amount may be obtained by dividing the second total historical data amount by the third time span.
In operation S702, a standard data capacity of the third type of storage space is generated according to the average data amount and a preset time threshold of the third type of storage space.
According to an embodiment of the present disclosure, the third category of storage space may generally store only a preset number of days of data volume, for example, may store only the last three days of data volume.
According to the embodiment of the present disclosure, the third type of storage space may store data amounts of monday, tuesday and wednesday, for example, and may delete data of monday at zero point of wednesday, and start receiving and storing data of thursday, so as to ensure that only data amounts of near three days are stored in the third type of storage space.
According to an embodiment of the present disclosure, the preset time threshold may include a number of days that the third category table space may store data.
In operation S703, an estimated growth data amount is obtained according to the standard data capacity and a second rated data capacity of the third type of storage space.
According to an embodiment of the present disclosure, the above-described operations S701 to S703 may be represented by the following formula (2).
Figure BDA0003164073300000131
Wherein x is2May represent a second amount of historical data, d1May represent a third time span, d2May represent a preset time threshold, Y2May represent an estimated growth data volume.
FIG. 8 is a flow diagram illustrating generation of a volume expansion script based on an estimated growth data volume according to an embodiment of the present disclosure.
As shown in fig. 8, the generating of the capacity expansion script according to the estimated growth data amount according to the embodiment includes operations S801 to S803.
In operation S801, N data files are obtained according to the estimated growth data amount, where a sum of the data amounts of the N data files is greater than or equal to the estimated growth data amount, and N is an integer.
In operation S802, identification information of each of the N data files is acquired.
In operation S803, the identification information is used as a script parameter to generate a capacity expansion script.
According to embodiments of the present disclosure, the amount of data amplification of a database by a single data file is generally limited, for example, one data file may amplify the amount of data of 30GB of the database.
According to the embodiment of the disclosure, the number of the required data files can be obtained by calculating the ratio of the estimated growth data volume and the amplification data volume of the single data file to the database.
According to an embodiment of the present disclosure, the identification information may include, for example, a storage path of the data file, a file name of the data file, and the like.
Based on the above database capacity expansion method, the present disclosure also provides a database capacity expansion device. The apparatus will be described in detail below with reference to fig. 9.
Fig. 9 is a block diagram schematically illustrating a structure of a database capacity expansion apparatus according to an embodiment of the present disclosure.
As shown in fig. 9, the database capacity expansion apparatus 900 of this embodiment includes an obtaining module 901, an estimating module 902, a generating module 903, and a capacity expansion module 904.
The obtaining module 901 is configured to obtain history storage information of the database through an interface calling manner. In an embodiment, the obtaining module 901 may be configured to perform the operation S201 described above, which is not described herein again.
The estimation module 902 is configured to obtain an estimated increase data volume of the database in a preset management period according to the history storage information. In an embodiment, the estimation module 902 may be configured to perform the operation S202 described above, which is not described herein again.
The generating module 903 is configured to generate a capacity expansion script according to the estimated growth data amount. In an embodiment, the generating module 903 may be configured to perform the operation S203 described above, which is not described herein again.
The capacity expansion module 904 is configured to expand the database using the capacity expansion script. In an embodiment, the capacity expansion module 904 may be configured to perform the operation S204 described above, which is not described herein again.
According to the embodiment of the disclosure, the history storage information includes a first data volume and a first history data volume of the first class storage space, where the first data volume includes a data volume of the first class storage space at a first time point to be expanded, and the first history data volume includes a data volume of the first class storage space at a second time point having a first time span with the first time point to be expanded, where the first time span is equal to the preset management period.
According to an embodiment of the present disclosure, the estimation module 902 includes a first obtaining unit and a generating unit.
The first obtaining unit is used for obtaining a first storage threshold value of the first category of storage space.
And the generating unit is used for obtaining the estimated increase data volume of the first class storage space in the preset management period according to the first storage threshold, the first data volume and the first historical data volume.
According to an embodiment of the present disclosure, the generation unit includes a first generation subunit, a second generation subunit, and a third generation subunit.
And the first generation subunit is used for generating a first difference value according to the first data volume and the first historical data volume.
And the second generation subunit is used for generating a first ratio according to the first difference and the first storage threshold.
And the third generation subunit is used for obtaining the estimated increase data volume of the first class storage space in the preset management period according to the first ratio and the first data volume.
According to an embodiment of the present disclosure, the database capacity expansion apparatus 900 further includes a first obtaining module, a first generating module, a second generating module, and a third generating module.
The first obtaining module is used for obtaining a first rated capacity and a second storage threshold of the first category of storage space.
And the first generation module is used for generating a second ratio according to the first rated capacity and a second storage threshold value.
And the second generation module is used for generating the total estimated data amount according to the first data amount and the estimated growth data amount.
And the third generation module is used for generating a capacity expansion script according to the estimated increase data volume under the condition that the estimated data total volume is less than or equal to the second ratio.
According to the embodiment of the disclosure, the history storage information comprises a first history data total amount of the second category storage space, wherein the first history data total amount comprises a data total amount of the second category storage space in a history management period having a second time span with a preset management period; the second category of storage space has an initial storage capacity.
According to an embodiment of the present disclosure, the estimation module 902 includes a first generation unit and a determination unit.
The first generation unit is used for generating a second difference value according to the total amount of the first historical data and the initial storage capacity;
and the determining unit is used for taking the second difference value as the estimated growth data amount.
According to the embodiment of the disclosure, the history storage information includes a second data volume and a second history data total volume of the third category storage space, where the second data volume includes a data volume of the third category storage space at the second time point to be expanded, and the second history data total volume includes a data total volume of the third category storage space in a history management period having a third time span with the second time point to be expanded.
According to the embodiment of the present disclosure, the estimation module 902 further includes a second generation unit, a third generation unit, and a second determination unit.
The second generation unit is used for generating an average data volume according to the second historical data total volume and the third time span;
the third generation unit is used for generating the standard data capacity of the third-class storage space according to the average data volume and the preset time threshold of the third-class storage space;
and the second determining unit is used for obtaining the estimated growth data volume according to the standard data volume and the second rated data volume of the third type storage space.
According to an embodiment of the present disclosure, the generating module 903 includes a first acquiring unit, a second acquiring unit, and a fourth generating unit.
The first obtaining unit is used for obtaining N data files according to the estimated growth data quantity, wherein the sum of the data quantities of the N data files is larger than or equal to the estimated growth data quantity, and N is an integer;
the second acquisition unit is used for acquiring the identification information of each data file in the N data files;
and the fourth generation unit is used for generating the capacity expansion script by taking the identification information as a script parameter.
According to the embodiment of the present disclosure, any multiple modules of the obtaining module 901, the estimating module 902, the generating module 903 and the capacity expanding module 904 may be combined into one module to be implemented, or any one module may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the obtaining module 901, the estimating module 902, the generating module 903, and the capacity expanding module 904 may be implemented at least partially as a hardware circuit, for example, a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementation manners of software, hardware, and firmware, or implemented by a suitable combination of any several of them. Alternatively, at least one of the obtaining module 901, the estimating module 902, the generating module 903 and the capacity expanding module 904 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.
Fig. 10 schematically illustrates a block diagram of an electronic device suitable for implementing a database capacity expansion method according to an embodiment of the present disclosure.
As shown in fig. 10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. Processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the present disclosure.
In the RAM1003, various programs and data necessary for the operation of the electronic apparatus 1000 are stored. The processor 1001, ROM1002, and RAM1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM1002 and/or the RAM 1003. Note that the programs may also be stored in one or more memories other than the ROM1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
Electronic device 1000 may also include an input/output (I/O) interface 1005, the input/output (I/O) interface 1005 also being connected to bus 1004, according to an embodiment of the present disclosure. Electronic device 1000 may also include one or more of the following components connected to I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM1002 and/or the RAM1003 described above and/or one or more memories other than the ROM1002 and the RAM 1003.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the database capacity expansion method provided by the embodiment of the disclosure.
The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 1001. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication part 1009, and/or installed from the removable medium 1011. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. The computer program performs the above-described functions defined in the system of the embodiment of the present disclosure when executed by the processor 1001. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims (11)

1. A database capacity expansion method comprises the following steps:
acquiring historical storage information of a database in an interface calling mode;
obtaining the estimated growth data volume of the database in a preset management period according to the historical storage information;
generating a capacity expansion script according to the estimated increase data volume;
and expanding the capacity of the database by using the capacity expansion script.
2. The method according to claim 1, wherein the historical storage information includes a first data amount of a first category storage space and a first historical data amount, wherein the first data amount includes a data amount of the first category storage space at a first time point to be expanded, and the first historical data amount includes a data amount of the first category storage space at a second time point having a first time span with the first time point to be expanded, wherein the first time span is equal to the preset management period;
the obtaining of the estimated growth data volume of the database in a preset management period according to the historical storage information comprises:
acquiring a first storage threshold value of the first category storage space;
and obtaining the estimated increase data volume of the first class storage space in a preset management period according to the first storage threshold, the first data volume and the first historical data volume.
3. The method according to claim 2, wherein the obtaining of the estimated increase data volume of the first class storage space in a preset management period according to the first storage threshold, the first data volume and the first historical data volume comprises:
generating a first difference value according to the first data volume and the first historical data volume;
and obtaining the estimated increase data volume of the first class storage space in a preset management period according to the first difference and the first storage threshold.
4. The method according to claim 2 or 3, after obtaining the estimated increase data amount of the first class storage space in a preset management period, the method further comprising:
acquiring a first rated capacity of the first category of storage space;
generating a maximum safe storage amount according to the first rated capacity and the first storage threshold value;
generating a total estimated data amount according to the first data amount and the estimated growth data amount;
and under the condition that the estimated data total amount is less than or equal to the maximum safe storage amount, generating a capacity expansion script according to the estimated increase data amount.
5. The method of claim 1, wherein the historical storage information comprises a first historical data total for a second category of storage space, wherein the first historical data total comprises a data total for the second category of storage space over a historical management period having a second time span with the preset management period; the second category of storage space has an initial storage capacity;
the obtaining of the estimated growth data volume of the database in a preset management period according to the historical storage information comprises:
generating a second difference value according to the first historical data total amount and the initial storage capacity;
and taking the second difference value as the estimated growth data amount.
6. The method of claim 1, wherein the historical storage information includes a second data volume of a third category of storage space and a second historical data volume, wherein the second data volume includes a data volume of the third category of storage space at a second time point to be expanded, and the second historical data volume includes a data volume of the third category of storage space in a historical management period having a third time span with the second time point to be expanded;
the obtaining of the estimated growth data volume of the database in a preset management period according to the historical storage information comprises:
generating an average data volume according to the second historical data total and the third time span;
generating a standard data capacity of the third-class storage space according to the average data volume and a preset time threshold of the third-class storage space;
and obtaining the estimated growth data volume according to the standard data volume and the second rated data volume of the third type of storage space.
7. The method of claim 1, wherein generating a capacity expansion script according to the pre-estimated growth data amount comprises:
acquiring N data files according to the estimated growth data volume, wherein the sum of the data volumes of the N data files is greater than or equal to the estimated growth data volume, and N is an integer;
acquiring identification information of each data file in the N data files;
and using the identification information as a script parameter to generate the capacity expansion script.
8. A database capacity expansion apparatus comprising:
the acquisition module is used for acquiring historical storage information of the database in an interface calling mode;
the estimation module is used for obtaining the estimated increase data volume of the database in a preset management period according to the historical storage information;
the generating module is used for generating a capacity expansion script according to the estimated increase data volume; and
and the capacity expansion module is used for expanding the capacity of the database by using the capacity expansion script.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.
10. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.
11. A computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of claims 1 to 7.
CN202110803627.0A 2021-07-15 2021-07-15 Database capacity expansion method and device Pending CN113535726A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110803627.0A CN113535726A (en) 2021-07-15 2021-07-15 Database capacity expansion method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110803627.0A CN113535726A (en) 2021-07-15 2021-07-15 Database capacity expansion method and device

Publications (1)

Publication Number Publication Date
CN113535726A true CN113535726A (en) 2021-10-22

Family

ID=78128270

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110803627.0A Pending CN113535726A (en) 2021-07-15 2021-07-15 Database capacity expansion method and device

Country Status (1)

Country Link
CN (1) CN113535726A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115757363A (en) * 2023-01-06 2023-03-07 速度时空信息科技股份有限公司 Multi-level management method and system for three-dimensional cadastral database
CN116483886A (en) * 2023-04-10 2023-07-25 上海沄熹科技有限公司 Method for inquiring OLAP by combining KV storage engine and time sequence storage engine

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115757363A (en) * 2023-01-06 2023-03-07 速度时空信息科技股份有限公司 Multi-level management method and system for three-dimensional cadastral database
CN115757363B (en) * 2023-01-06 2023-05-12 速度时空信息科技股份有限公司 Multi-level management method and system for three-dimensional cadastral database
CN116483886A (en) * 2023-04-10 2023-07-25 上海沄熹科技有限公司 Method for inquiring OLAP by combining KV storage engine and time sequence storage engine
CN116483886B (en) * 2023-04-10 2024-04-02 上海沄熹科技有限公司 Method for inquiring OLAP by combining KV storage engine and time sequence storage engine

Similar Documents

Publication Publication Date Title
US11755452B2 (en) Log data collection method based on log data generated by container in application container environment, log data collection device, storage medium, and log data collection system
CN113535726A (en) Database capacity expansion method and device
CN113505302A (en) Method, device and system for supporting dynamic acquisition of buried point data and electronic equipment
CN107426336B (en) Method and device for adjusting push message opening rate
CN113076224B (en) Data backup method, data backup system, electronic device and readable storage medium
CN111258988A (en) Asset management method, device, electronic device, and medium
CN114003659A (en) Data synchronization method, data synchronization device, electronic equipment, storage medium and program product
US11108784B2 (en) Permission aggregator
CN114218283A (en) Abnormality detection method, apparatus, device, and medium
CN115408297A (en) Test method, device, equipment and medium
CN114443663A (en) Data table processing method, device, equipment and medium
CN114201508A (en) Data processing method, data processing apparatus, electronic device, and storage medium
CN114140091A (en) Operation record display method, device, equipment and medium
CN113761415A (en) Page display method and device
CN114268558B (en) Method, device, equipment and medium for generating monitoring graph
CN115312208B (en) Method, device, equipment and medium for displaying treatment data
CN114844810B (en) Heartbeat data processing method, device, equipment and medium
CN115987782B (en) Cloud hostname generation method, device, equipment, storage medium and program product
CN113362097B (en) User determination method and device
CN114201345A (en) Data processing method, device, electronic equipment and medium
CN117056340A (en) Account checking data processing method, device, equipment and storage medium
CN113419922A (en) Method and device for processing batch job running data of host
CN114218160A (en) Log processing method and device, electronic equipment and medium
CN116663928A (en) Operation control method, device and equipment for platform batch operation and storage medium
CN114328096A (en) Index monitoring method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination