CN111125146A - Report generation method and device and storage medium - Google Patents
Report generation method and device and storage medium Download PDFInfo
- Publication number
- CN111125146A CN111125146A CN201911182873.8A CN201911182873A CN111125146A CN 111125146 A CN111125146 A CN 111125146A CN 201911182873 A CN201911182873 A CN 201911182873A CN 111125146 A CN111125146 A CN 111125146A
- Authority
- CN
- China
- Prior art keywords
- task
- report
- library
- node
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2428—Query predicate definition using graphical user interfaces, including menus and forms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the application discloses a report generation method, a report generation device and a storage medium, wherein the method comprises the following steps: when the first task is detected to be started, dividing the first task into at least one second task based on the number of nodes of the production library, wherein one second task corresponds to one node; for each second task, obtaining a number segment list corresponding to the node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments; and inquiring the report data of each sub-number segment from the production library and storing the report data in a report library. The method includes the steps that a two-dimensional task decomposition method is adopted to store report data in a production library into a report library, and two-stage task decomposition is carried out, so that the processed data volume is reduced and smaller, the system data processing efficiency is improved, and the data processing time is saved. And then, when a report generation instruction is detected, generating a report based on the report data stored in the report library, thereby improving the report generation efficiency.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for generating a report, and a storage medium.
Background
With the continuous development of domestic telecommunication enterprises, the competition of the telecommunication industry tends to be white. On the one hand, the scope of customers for selecting telecommunication services and telecommunication enterprises is getting bigger and bigger, and the competition among the telecommunication enterprises for customers is also getting stronger and stronger. On the other hand, telecommunication customers are growing at a high speed in recent years, and a large customer group with large demand difference is formed; meanwhile, various new services suitable for different crowds are continuously released, and telecommunication enterprises need to promote the most appropriate service to the most needed customers by subdividing markets and customer groups, so that the best matching between the service and the customers is realized.
Based on the background, technical personnel provide a number card resource report system, and provide an operation decision basis for enterprise decision makers and managers at all levels through analyzing and mining daily operation data of telecommunication enterprises so as to realize refined marketing.
However, the number card resource data is huge and reaches hundreds of millions, and the report system of the traditional relational database adopts a single thread to process data, so that the data processing efficiency is low, the report generation time is long, the report updating period is long, and the timeliness is poor.
Disclosure of Invention
The embodiment of the application provides a report generation method, a report generation device and a storage medium, so as to generate a report quickly.
In a first aspect, an embodiment of the present application provides a report generating method, including:
when the first task is detected to be started, dividing the first task into at least one second task based on the number of nodes of a production library, wherein one second task corresponds to one node;
for each second task, obtaining a number segment list corresponding to a node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments, wherein report data of each number in one number segment list is stored under each node;
inquiring report data of each sub-number segment from the production library and storing the report data in a report library;
and when a report generation instruction is detected, generating a report based on the report data stored in the report library.
In a possible implementation manner of the first aspect, before detecting that the first task is started, the method further includes:
acquiring a timing task;
dividing the timing task into a plurality of first tasks according to a report type preset by the timing task, and allocating a timing trigger to each first task;
detecting a first task initiation comprising:
and detecting the starting of the first task by detecting a timing trigger corresponding to the first task.
In a possible implementation manner of the first aspect, the detecting that the first task is started by detecting a timing trigger corresponding to the first task includes:
and receiving a starting instruction sent by a timing trigger corresponding to the first task when the starting time corresponding to the first task is reached, wherein the starting instruction is used for indicating the first task to start.
Optionally, the starting time corresponding to different first tasks is different.
Optionally, the timing task includes: number timing tasks and card timing tasks.
In a possible implementation manner of the first aspect, before obtaining, from the node corresponding to the second task, the number segment list corresponding to the node, the method further includes:
storing each second task into a task queue, and allocating a thread to each second task;
obtaining a number segment list corresponding to the node from the node corresponding to the second task, including:
and controlling the thread corresponding to the second task, and obtaining the number segment list corresponding to the node from the node corresponding to the second task.
In a possible implementation manner of the first aspect, the querying the report data of each sub-number segment from the production library includes:
and for each second task, controlling the thread of the second task, and obtaining report data of each sub-number segment corresponding to the second task from a node corresponding to the second task.
In a possible implementation manner of the first aspect, generating a report based on report data stored in the report library when a report generation instruction is detected includes:
acquiring a report generation instruction input by a user, wherein the report generation instruction is used for indicating generation of a report meeting the condition parameter information;
and inquiring report data meeting the condition parameter information from the report library according to the condition parameter information to generate a report.
Optionally, the production library is a distributed database, the production library includes a plurality of nodes, and report data of different number segment lists are stored in different nodes.
In a second aspect, an embodiment of the present application provides a report generating apparatus, including:
the detection module is used for detecting whether the first task is started or not;
the first dividing module is used for dividing the first task into at least one second task based on the number of nodes of the production library when the detection module detects that the first task is started, wherein one second task corresponds to one node;
the processing module is used for acquiring a number segment list corresponding to the node from the node corresponding to each second task, wherein report data of each number in one number segment list is stored under each node;
a second dividing module, configured to divide the number segment list into a plurality of sub-number segments;
the query module is used for querying the report data of each sub-number segment from the production library and storing the report data into a report library;
and the generating module is used for generating a report based on the report data stored in the report library when a report generating instruction is detected.
In a possible implementation manner of the second aspect, the apparatus further includes:
the acquisition module is used for acquiring the timing task;
the third division module is used for dividing the timing task into a plurality of first tasks according to the report type preset by the timing task and distributing a timing trigger to each first task;
the detection module is specifically configured to detect a timing trigger corresponding to the first task to detect that the first task is started.
In a possible implementation manner of the second aspect, the detecting module is configured to receive a start instruction sent by a timing trigger corresponding to the first task when a start time corresponding to the first task arrives, where the start instruction is used to instruct the first task to start.
Optionally, the starting time corresponding to different first tasks is different.
Optionally, the timing task includes: number timing tasks and card timing tasks.
In a possible implementation manner of the second aspect, the apparatus further includes:
the configuration module is used for storing each second task into a task queue and distributing a thread to each second task;
the processing module is specifically configured to control a thread corresponding to the second task, and obtain a number segment list corresponding to a node from the node corresponding to the second task.
In a possible implementation manner of the second aspect, the query module is specifically configured to control, for each second task, a thread of the second task, and obtain, from a node corresponding to the second task, report data of each sub-number segment corresponding to the second task.
In a possible implementation manner of the second aspect, the generating module includes:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a report generation instruction input by a user, and the report generation instruction is used for indicating generation of a report meeting condition parameter information;
and the query unit is used for querying the report data meeting the condition parameter information from the report library according to the condition parameter information to generate a report.
Optionally, the production library is a distributed database, the production library includes a plurality of nodes, and report data of different number segment lists are stored in different nodes.
In a third aspect, an embodiment of the present application provides an electronic device, including:
a memory for storing a computer program;
the processor is configured to execute the computer program, and in particular, to execute the report generation method according to the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer storage medium, where the storage medium includes computer instructions, and when the instructions are executed by a computer, the computer is caused to implement the report generation method according to the first aspect of the present application.
In a fifth aspect, the present application provides a computer program product, where the program product includes a computer program, where the computer program is stored in a readable storage medium, and at least one processor of a computer can read the computer program from the readable storage medium, and the at least one processor executes the computer program to make the computer implement the report generation method according to any one of the first aspect.
According to the report generation method, the report generation device and the storage medium, when the starting of a first task is detected, the first task is divided into at least one second task based on the number of nodes of a production library, wherein one second task corresponds to one node; for each second task, obtaining a number segment list corresponding to a node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments, wherein report data of each number in one number segment list is stored under each node; and inquiring the report data of each sub-number segment from the production library and storing the report data in a report library. The method of the embodiment of the application stores the report data in the production library into the report library, and performs two-dimensional task decomposition in the storage process, namely, a first task is decomposed into a plurality of second tasks based on the nodes of the production library, and then the second tasks are decomposed into a plurality of sub-number segment tasks, and through two-stage task decomposition, the processed data volume is reduced and smaller, so that the system data processing efficiency is improved, and the data processing time is saved. And then, when a report generation instruction is detected, generating a report based on the report data stored in the report library, namely directly inquiring the report data from the report library instead of inquiring the report data from the production library, thereby further improving the report efficiency.
Drawings
FIG. 1 is a schematic illustration of a production library to which embodiments of the present application relate;
FIG. 2 is a schematic diagram of a report generation system according to an embodiment of the present application;
fig. 3 is a flowchart of a report generation method provided in the embodiment of the present application;
FIG. 4 is a diagram illustrating report data stored in a production library according to an embodiment of the present disclosure;
FIG. 5 is an exploded view of a task involved in an embodiment of the present application;
fig. 6 is a flowchart of a report generation method provided in the embodiment of the present application;
fig. 7 is a flowchart of a report generation method provided in the embodiment of the present application;
FIG. 8 is a diagram illustrating an example of a task queue according to an embodiment of the present application;
fig. 9 is a flowchart of a report generation method provided in the embodiment of the present application;
fig. 10 is a schematic diagram of a report generating apparatus according to an embodiment of the present application;
fig. 11 is a schematic diagram of a report generating apparatus according to an embodiment of the present application;
fig. 12 is a schematic diagram of a report generating apparatus according to an embodiment of the present application;
fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solution of the present application better understood, the related art related to the present application will be first described.
Distributed Database (DRDS) is a logically unified Database formed by connecting a plurality of physically Distributed data storage units by using a high-speed computer network, and storing data in an original centralized Database in a Distributed manner on a plurality of data storage nodes connected through a network to obtain a larger storage capacity and a higher concurrent access amount. Compared with a centralized database, the distributed database has the advantages that data of the distributed database is stored in more than two nodes in a network, each node has independent processing capacity, and the nodes can transmit the data through network communication to cooperatively complete cross-node processing tasks. As shown in fig. 1, the distributed database has 8 nodes, i.e. 8 libraries in the graph, each library is a node, each node has independent processing capability, and the nodes do not interfere with each other.
A task queue, which generally refers to a container for maintaining a series of tasks, requires a medium for storing data such as the name of the task, the type of the task, parameters required by the task, and the task itself, and can be understood as a storage medium. The method abstracts the inter-thread communication, limits that the inter-thread can only pass through the transmission task, and related data and operation are stored by the task. A task queue is defined as a thread-safe first-in-first-out queue that is capable of passing tasks encapsulating data and operations between multiple threads. The task queue is used for executing functions with slow processing time or functions with delayed network requests in a queuing mode.
Fig. 2 is a schematic diagram of a report generating system according to an embodiment of the present application, and as shown in fig. 2, the report generating system includes functions, such as a report display platform, a report application platform, and a report Extraction Transformation Load (ETL) platform, which are executed by a processor in an electronic device.
The report display platform mainly comprises report display functions of a number report, a smart card report, an Embedded Subscriber Identity Module (ESIM) service report and other major services, provides a free screening function, and can flexibly screen data for a field or a plurality of fields according to user requirements. Meanwhile, the report display platform supports a user-defined report template, and can set a display field for exporting the report, thereby fully meeting the individual requirements of users.
The report application platform mainly comprises a functional interface service, a service interface service, a mapping service, a database interface and the like of a High Speed Frame (HSF). The database interface is used as a storage layer and is responsible for the connection work of the report application platform and the database, the number card resource report data is directly read from the report library, and the number card resource report data in the report library is generated by processing of a report Extraction Transformation Loading (ETL) platform. The mapping service serves as a persistent layer and is responsible for encapsulating various operations on the database required by the report application platform. The service interface service is used as a service layer and is responsible for the specific implementation of application logic, including entry processing, rule checking, authority checking, query logic, exit processing and the like, and simultaneously provides interface calling capability for functional interface services and other systems. The functional interface service is used as a functional layer and is responsible for secondary packaging of the business interface service, and an interface is provided for the outside for the report display platform to call.
The report extraction, transformation and loading (ETL) platform is a core part of the system and is responsible for the process of extracting, transposing and loading mass number card resource data from the data of each production library into a report library. The system mainly comprises a timed task module, a task queue module, a listener module, a multithreading module, a data queue module, a file import module and the like.
It should be understood that, in the present embodiment, "B corresponding to a" means that B is associated with a. In one implementation, B may be determined from a. It should also be understood that determining B from a does not mean determining B from a alone, but may be determined from a and/or other information.
In the description of the present application, "plurality" means two or more than two unless otherwise specified.
In addition, in order to facilitate clear description of technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items having substantially the same functions and actions. Those skilled in the art will appreciate that the terms "first," "second," etc. do not denote any order or quantity, nor do the terms "first," "second," etc. denote any order or importance.
The report generation method provided by the embodiment of the present application is described in detail below with specific examples.
Fig. 3 is a flowchart of a report generation method provided in the embodiment of the present application, and as shown in fig. 3, the method in the embodiment of the present application includes:
s101, when the first task is detected to be started, dividing the first task into at least one second task based on the number of nodes of the production library, wherein one second task corresponds to one node.
The execution subject of the embodiment of the present application is an apparatus with a report generating function, for example, the apparatus is a report generating apparatus, and the apparatus may be a single electronic device, for example, a terminal device such as a PC, a computer, a tablet, a mobile phone, and the like, or a component in the electronic device, for example, a processor in the electronic device.
The production library described in the embodiment of the present application stores report data of a large number card, and the report extraction conversion loading (ETL) platform shown in fig. 2 may communicate with the production library, and may perform operations such as extraction, transposition, and loading on data in the production library.
Optionally, as shown in fig. 1, the production library in the embodiment of the present application is a distributed database, and includes a plurality of nodes, and report data of a part of number cards is stored in each node.
In an example, fig. 4 is a schematic diagram illustrating report data stored in a production library according to an embodiment of the present application. As shown in fig. 4, the data in the production library is calculated according to the value of the split field and a specific algorithm, and the data is stored in the corresponding fragment according to the calculation result, where each fragment has a corresponding library name node. Optionally, as shown in fig. 4, in the embodiment of the present application, the number segment NUM _ H3 is used as a sub-library key, i.e., a split field, to perform sub-library storage, for example, the number 13001230002 is stored in the library 2 according to the first 7 bits 1300123 of the number, for example, the number 15540750005 is stored in the library 1 according to the first 7 bits 1554075 of the number, so as to implement sub-library storage of number data.
As shown in fig. 4, report data of different number segment lists are stored in different nodes of the production library according to the embodiment of the present application, for example, library 2 stores number data of number segment list 1300123, and library 1 stores number data of number segment list 1554075.
The first task may be understood as a task of reading the number or the card report data from the production library to the report library.
The method and the device for processing the report data have the advantages that the starting time is set for each first task, when the starting time is up, the first task is started, and the report data corresponding to the first task are read from the production library to the report library.
Optionally, because the data size of the report data corresponding to the first task is large, in order to quickly obtain the report data of the first task, the granularity of the first task is refined.
Specifically, the first task is refined into at least one second task based on the number of nodes of the production library. For example, as shown in fig. 1, the production library includes 8 nodes, so that the first task can be divided into 8 second tasks, where one second task corresponds to one node.
S102, aiming at each second task, obtaining a number segment list corresponding to the node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments, wherein report data of each number in one number segment list is stored under each node.
According to the embodiment of the application, the second tasks can be executed in parallel, and the efficiency of acquiring the report data corresponding to the first task from the production library can be improved.
The processing procedure of each second task is similar, and one second task is taken as an example for description, and other second tasks may be referred to.
In order to further improve the efficiency of obtaining the report data of the first task, the second task is refined again.
Specifically, for each second task, a number segment list stored under the node is obtained from the node corresponding to the second task. For example, referring to the above example, for the second task 1, the node 1 corresponding to the second task 1 stores the report data of the number segment list 1554075 in the node 1 (i.e., library 1). With reference to the above method, a list of number segments corresponding to each second task may be obtained.
Then, the number segment list corresponding to each second task is divided into a plurality of sub-number segments.
In an example, the number segment list corresponding to the second task is divided into a plurality of sub-number segments according to a preset number of packets, for example, every 500 number segments form a group, and the number segment list is divided into a plurality of sub-number segments.
In another example, according to a preset grouping rule, the number segment list corresponding to the second task is divided into a plurality of sub-number segments, for example, numbers in the number segment list are averagely divided into a plurality of sub-number segments.
Optionally, in this embodiment of the application, the number segment list corresponding to the second task may be further divided into a plurality of sub-number segments based on other division rules.
S103, inquiring the report data of each sub-number segment from the production library and storing the report data in a report library.
According to the steps, a plurality of sub-number segments corresponding to each second task can be obtained, then, the sub-number segments are used as query units, the report data of each sub-number segment are queried from the production library, and the queried report data are stored in the report library.
It should be noted that, in the query of the embodiment of the present application, the multiple sub-number segments of each second task are executed in parallel in the query process, so that the query efficiency of the report data can be greatly improved.
The above S101 to S103 are processes related to the embodiment of the present application for reading the report data in the production library into the report library.
According to the method, before the report query task of the user is received, the report data is firstly read from the production library to the report library, and in the subsequent query process, the user can directly query the report data from the report library without querying the report data from the production library, so that the efficiency of querying the report data by the user is greatly improved.
As can be seen from the above, in the embodiment of the present application, a two-dimensional task decomposition method is adopted to convert an original one-dimensional queue into a two-dimensional queue, as shown in fig. 5, where one dimension is to perform task decomposition according to the number of nodes in a production library, that is, a first task is decomposed into a plurality of second tasks; and the other dimension is to perform task decomposition according to the number of the number segment lists, namely to divide the number segment list corresponding to the second task into a plurality of sub-number segments. Therefore, through two task decompositions, a single task is more unitized, and the data volume to be processed is smaller, so that the system data processing efficiency is improved, and the data processing time is saved.
Optionally, the steps from S101 to S103 may be performed by a report Extraction Transformation Loading (ETL) platform shown in fig. 2.
And S104, when a report generation instruction is detected, generating a report based on the report data stored in the report library.
According to the steps, the report data can be read from the production library to the report library, so that when the report generation instruction is detected, the report data can be obtained from the report library based on the report generation instruction, and then the report is generated.
In one practical application, the user may input the report generation instruction on the report presenting platform shown in fig. 2, for example, the report generation instruction is input in the search bar of the report presenting platform. And the report display platform sends the report generation instruction to the report application platform, each module in the report application platform reads report data meeting the report generation instruction from the production library, processes the obtained report data, and generates a report and returns the generated report to the report display platform. And the report display platform displays the report returned by the report application platform to the user.
According to the report generation method provided by the embodiment of the application, when the starting of a first task is detected, the first task is divided into at least one second task based on the number of nodes of a production library, wherein one second task corresponds to one node; for each second task, obtaining a number segment list corresponding to a node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments, wherein report data of each number in one number segment list is stored under each node; and inquiring the report data of each sub-number segment from the production library and storing the report data in a report library. The method of the embodiment of the application stores the report data in the production library into the report library, and performs two-dimensional task decomposition in the storage process, namely, a first task is decomposed into a plurality of second tasks based on the nodes of the production library, and then the second tasks are decomposed into a plurality of sub-number segment tasks, and through two-stage task decomposition, the processed data volume is reduced and smaller, so that the system data processing efficiency is improved, and the data processing time is saved. And then, when a report generation instruction is detected, generating a report based on the report data stored in the report library, namely directly inquiring the report data from the report library instead of inquiring the report data from the production library, thereby further improving the report efficiency.
Fig. 6 is a flowchart of a report generation method provided in the embodiment of the present application, where on the basis of the above embodiment, as shown in fig. 6, the method in the embodiment of the present application includes:
s201, acquiring a timing task.
Optionally, the timing task includes: number timing tasks and card timing tasks.
S202, dividing the timing task into a plurality of first tasks according to a report type preset by the timing task, and allocating a timing trigger to each first task.
The embodiment of the application further includes steps S201 and S202 before detecting the start of the first task.
Specifically, a timing task is obtained, and the timing task may be a number timing task or a card timing task, where the card timing task includes a smart card timing task, an ESIM card timing task, and the like. Then, according to the report type preset by the timing task, the timing task is divided into a plurality of first tasks, for example, the timing task is a number timing task, the number of the preset report types of the number timing task is 6, so that the number timing task can be divided into 6 first tasks, and each first task corresponds to one report type. Then, a timing trigger is assigned to each first task.
Alternatively, the timing trigger may be a Quartz (an open source job scheduling framework written entirely by java) trigger.
S203, detecting the starting of the first task by detecting a timing trigger corresponding to the first task.
In this case, the step of S203 may be to detect whether the first task is started by detecting a timing trigger of the first task in S101.
In a possible implementation manner, in the step S202, detecting the start of the first task by detecting a timing trigger corresponding to the first task, includes the steps of:
and step A, receiving a starting instruction sent by a timing trigger corresponding to the first task when the starting time corresponding to the first task is reached, wherein the starting instruction is used for indicating the first task to start.
Optionally, the starting times corresponding to different first tasks are different, that is, due to the limitation of data processing resources, the first tasks are executed in a staggered manner according to the preset trigger time, rather than in parallel.
S204, dividing the first task into at least one second task based on the number of the nodes of the production library, wherein one second task corresponds to one node.
S205, aiming at each second task, obtaining a number segment list corresponding to the node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments, wherein report data of each number in one number segment list is stored under each node.
S206, inquiring the report data of each sub-number segment from the production library and storing the report data in a report library.
And S207, when a report generation instruction is detected, generating a report based on the report data stored in the report library.
The specific execution process of S204 to S207 may refer to the description of S101 to S104, which is not repeated herein.
According to the report generation method provided by the embodiment of the application, the acquired timed task is divided into the plurality of first tasks through the report type preset by the timed task, and one timed trigger is allocated to each first task, so that whether the first task is started or not can be detected by detecting the timed trigger of the first task. Namely, according to the embodiment of the application, the timed task is divided into the plurality of first tasks, the report data corresponding to the first tasks are inquired from the production library, the inquiry granularity of the timed task is reduced, the starting time of each first task is executed in a staggered manner, the system congestion caused when the plurality of first tasks are executed simultaneously is avoided, and the inquiry efficiency of the report data is further improved. Meanwhile, a timing trigger is configured for each first task, so that whether the first task is started or not can be accurately detected.
Fig. 7 is a flowchart of a report generation method provided in the embodiment of the present application, where on the basis of the foregoing embodiment, as shown in fig. 7, the method in the embodiment of the present application includes:
s301, acquiring a timing task.
S302, dividing the timing task into a plurality of first tasks according to a report type preset by the timing task, and allocating a timing trigger to each first task.
Optionally, the timing task includes: number timing tasks and card timing tasks.
S303, detecting the starting of the first task by detecting a timing trigger corresponding to the first task.
S304, dividing the first task into at least one second task based on the number of the nodes of the production library, wherein one second task corresponds to one node.
The specific execution process of S301 to S304 may refer to the description of S201 to S204, which is not repeated herein.
S305, storing each second task into a task queue, and allocating a thread to each second task.
As shown in fig. 8, in the embodiment of the present application, according to the number of nodes in the production library, a first task is divided into a plurality of second tasks, the second tasks are stored in a task queue, and a thread is allocated to each second task.
Optionally, as shown in fig. 8, the main thread stores the second task in the task queue, and it is assumed that N second tasks are stored in the task queue, each second task corresponds to one thread, and the thread corresponding to each second task can execute the second task.
S306, aiming at each second task, controlling a thread corresponding to the second task, obtaining a number segment list corresponding to a node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments.
Continuing to refer to fig. 8, each second task in the task queue is executed in parallel, and for one of the second tasks, it is assumed to be the second task 2, the thread corresponding to the second task 2 acquires the second task 2 from the task queue, and obtains the number segment list corresponding to the node 2 from the node (assumed to be the node 2) corresponding to the second task. Next, the number segment list corresponding to the second node 2 is divided into a plurality of sub-number segments, and the specific division process may refer to the description of the embodiment shown in fig. 3.
S307, inquiring the report data of each sub-number segment from the production library and storing the report data in a report library.
Specifically, for the sub-number segment corresponding to each second task, the thread corresponding to the second task is used to obtain the report data of each sub-number segment corresponding to the second task from the node corresponding to the second task, and the obtained report data is stored in the report library.
Optionally, in an example, the obtained report data may also be stored in a data list, and a listener may perform real-time monitoring on the data queue. And when the tasks to be processed in the data queue are found, importing the report data in the data queue into a report library.
And S308, when a report generation instruction is detected, generating a report based on the report data stored in the report library.
The specific execution process of S306 to S308 may refer to the description of S103 to S104, which is not repeated herein.
In one possible implementation, as shown in fig. 9, the step S308 includes:
s3081, a report generation instruction input by a user is obtained, and the report generation instruction is used for indicating generation of a report meeting the condition parameter information.
S3082, according to the condition parameter information, inquiring report data meeting the condition parameter information from the report library to generate a report.
For example, the condition parameter information may include: regional information, number segment information, customer information, report style, and the like. Thus, the user inputs the condition parameter information to the report display platform shown in fig. 2, the report display platform sends the condition parameter information to the report application platform, the report application platform obtains report data meeting the regional information, number segment information, customer information and the like from the report library, the report application platform makes the report data into a report meeting the report style according to the report style input by the user, the report data is returned to the report display platform, and the display report display platform displays the generated report to the user.
Optionally, the report display platform according to the embodiment of the present application further receives a modification operation of the user on the generated report, and modifies the generated report according to the modification operation of the user.
According to the report generation method provided by the embodiment of the application, the first task is divided into a plurality of second tasks according to the number of nodes of the production library, a thread is configured for each second task, the number segment list is obtained from the node corresponding to the second task through the thread of each second task, and the report data of each sub-number segment corresponding to the second task is obtained from the production library through the thread of the second task, so that the report data can be accurately obtained.
Fig. 10 is a schematic diagram of a report generating apparatus provided in an embodiment of the present application, where the report generating apparatus in the embodiment of the present application may be the electronic device described above, or may be a component (e.g., an integrated circuit, a chip, or the like) of the electronic device, and as shown in fig. 10, the report generating apparatus 100 may include:
a detecting module 110, configured to detect whether a first task is started;
a first dividing module 120, configured to divide, when the detection module detects that a first task is started, the first task into at least one second task based on the number of nodes in the production library, where one second task corresponds to one node;
a processing module 130, configured to obtain, for each second task, a number segment list corresponding to a node from the node corresponding to the second task, where report data of each number in one number segment list is stored under each node;
a second dividing module 140, configured to divide the number segment list into a plurality of sub-number segments;
the query module 150 is used for querying the report data of each sub-number segment from the production library and storing the report data into a report library;
the generating module 160 is configured to generate a report based on the report data stored in the report library when a report generating instruction is detected.
The report generation device of the embodiment of the application can be used for executing the technical scheme of the report generation method, the implementation principle and the technical effect are similar, and details are not repeated here.
Fig. 11 is a schematic view of a report generating apparatus provided in an embodiment of the present application, where on the basis of the foregoing embodiment, as shown in fig. 11, the apparatus in the embodiment of the present application further includes:
an obtaining module 170, configured to obtain a timing task;
a third dividing module 180, configured to divide the timed task into multiple first tasks according to a report type preset in the timed task, and allocate a timing trigger to each first task;
the detecting module 110 is specifically configured to detect a timing trigger corresponding to the first task to detect the start of the first task.
In a possible implementation manner, the detecting module 110 is configured to receive, when a start time corresponding to the first task arrives, a start instruction sent by a timing trigger corresponding to the first task, where the start instruction is used to instruct the first task to start.
Optionally, the starting time corresponding to different first tasks is different.
Optionally, the timing task includes: number timing tasks and card timing tasks.
The report generation device of the embodiment of the application can be used for executing the technical scheme of the report generation method, the implementation principle and the technical effect are similar, and details are not repeated here.
Fig. 12 is a schematic view of a report generating apparatus provided in an embodiment of the present application, where on the basis of the foregoing embodiment, as shown in fig. 12, the apparatus in the embodiment of the present application further includes:
the configuration module 190 is configured to store each second task into a task queue, and allocate a thread to each second task;
the processing module 130 is specifically configured to control the thread corresponding to the second task, and obtain the number segment list corresponding to the node from the node corresponding to the second task.
The query module 150 is specifically configured to control, for each second task, a thread of the second task, and obtain report data of each sub-number segment corresponding to the second task from a node corresponding to the second task.
Continuing to refer to fig. 11, the generating module 160 includes:
an obtaining unit 161, configured to obtain a report generation instruction input by a user, where the report generation instruction is used to instruct generation of a report meeting the condition parameter information;
the query unit 162 is configured to query, according to the conditional parameter information, report data meeting the conditional parameter information from the report library, and generate a report.
Optionally, the production library is a distributed database, the production library includes a plurality of nodes, and report data of different number segment lists are stored in different nodes.
The report generation device of the embodiment of the application can be used for executing the technical scheme of the report generation method, the implementation principle and the technical effect are similar, and details are not repeated here.
Alternatively, the report generation apparatus shown in fig. 10 to 12 may exist in a product form of a chip.
Fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 200 may implement the above method embodiments, and may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.
In one possible design, the electronic device 200 includes a processor 210 and a memory 220, and the processor 210 is configured to support the electronic device 200 to perform the corresponding functions of the above-mentioned method. The electronic device 200 may also include a memory 220, the memory 220 for coupling with the processor 210, which stores program instructions and data necessary for the electronic device 200.
When the electronic device 200 is turned on, the processor 210 can read the program instructions and data in the memory 220, interpret and execute the program instructions, and process the data of the program instructions.
Those skilled in the art will appreciate that fig. 13 shows only one memory 220 and one processor 210 for ease of illustration. In an actual electronic device 200, there may be multiple processors 210 and multiple memories 220. The memory 220 may also be referred to as a storage medium or a storage device, etc., which is not limited in this application.
The electronic device of the embodiment of the present application may be configured to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.
Based on such understanding, the technical solution of the present application may be substantially implemented or contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In addition, the method embodiments and the device embodiments may also refer to each other, and the same or corresponding contents in different embodiments may be referred to each other, which is not described in detail.
Claims (11)
1. A report generation method is characterized by comprising the following steps:
when the first task is detected to be started, dividing the first task into at least one second task based on the number of nodes of a production library, wherein one second task corresponds to one node;
for each second task, obtaining a number segment list corresponding to a node from the node corresponding to the second task, and dividing the number segment list into a plurality of sub-number segments, wherein report data of each number in one number segment list is stored under each node;
inquiring report data of each sub-number segment from the production library and storing the report data in a report library;
and when a report generation instruction is detected, generating a report based on the report data stored in the report library.
2. The method of claim 1, wherein before detecting the initiation of the first task, the method further comprises:
acquiring a timing task;
dividing the timing task into a plurality of first tasks according to a report type preset by the timing task, and allocating a timing trigger to each first task;
detecting a first task initiation comprising:
and detecting the starting of the first task by detecting a timing trigger corresponding to the first task.
3. The method of claim 2, wherein detecting the first task start by detecting a timing trigger corresponding to the first task comprises:
and receiving a starting instruction sent by a timing trigger corresponding to the first task when the starting time corresponding to the first task is reached, wherein the starting instruction is used for indicating the first task to start.
4. The method of claim 3, wherein different first tasks have different start-up times.
5. The method of claim 2, wherein the timing task comprises: number timing tasks and card timing tasks.
6. The method according to any of claims 1-5, wherein before obtaining the list of number segments corresponding to the node from the node corresponding to the second task, the method further comprises:
storing each second task into a task queue, and allocating a thread to each second task;
obtaining a number segment list corresponding to the node from the node corresponding to the second task, including:
and controlling the thread corresponding to the second task, and obtaining the number segment list corresponding to the node from the node corresponding to the second task.
7. The method of claim 6, wherein said querying said production library for reporting data for each of said sub-number segments comprises:
and for each second task, controlling the thread of the second task, and obtaining report data of each sub-number segment corresponding to the second task from a node corresponding to the second task.
8. The method according to any one of claims 1-5, wherein generating a report based on the report data stored in the report library upon detecting a report generation instruction comprises:
acquiring a report generation instruction input by a user, wherein the report generation instruction is used for indicating generation of a report meeting the condition parameter information;
and inquiring report data meeting the condition parameter information from the report library according to the condition parameter information to generate a report.
9. A report generation apparatus, comprising:
the detection module is used for detecting whether the first task is started or not;
the first dividing module is used for dividing the first task into at least one second task based on the number of nodes of the production library when the detection module detects that the first task is started, wherein one second task corresponds to one node;
the processing module is used for acquiring a number segment list corresponding to the node from the node corresponding to each second task, wherein report data of each number in one number segment list is stored under each node;
a second dividing module, configured to divide the number segment list into a plurality of sub-number segments;
the query module is used for querying the report data of each sub-number segment from the production library and storing the report data into a report library;
and the generating module is used for generating a report based on the report data stored in the report library when a report generating instruction is detected.
10. An electronic device, comprising:
a memory for storing a computer program;
the processor is configured to execute the computer program, in particular to execute the report generating method according to any of claims 1 to 8.
11. A computer storage medium characterized in that the storage medium comprises computer instructions which, when executed by a computer, cause the computer to implement the report generation method according to any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911182873.8A CN111125146B (en) | 2019-11-27 | 2019-11-27 | Report generation method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911182873.8A CN111125146B (en) | 2019-11-27 | 2019-11-27 | Report generation method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111125146A true CN111125146A (en) | 2020-05-08 |
| CN111125146B CN111125146B (en) | 2023-07-04 |
Family
ID=70496817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911182873.8A Active CN111125146B (en) | 2019-11-27 | 2019-11-27 | Report generation method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111125146B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103699618A (en) * | 2013-12-16 | 2014-04-02 | 广东威创视讯科技股份有限公司 | Data report generation method and system |
| US20140258209A1 (en) * | 2013-03-08 | 2014-09-11 | Platfora, Inc. | Systems and Methods for Interest-Driven Distributed Data Server Systems |
| CN107861933A (en) * | 2017-11-29 | 2018-03-30 | 北京百度网讯科技有限公司 | The method and apparatus for generating O＆M form |
| CN108733799A (en) * | 2018-05-17 | 2018-11-02 | 深圳市买买提信息科技有限公司 | A kind of method for processing report data, device and terminal |
| CN109145051A (en) * | 2018-07-03 | 2019-01-04 | 阿里巴巴集团控股有限公司 | The data summarization method and device and electronic equipment of distributed data base |
| US20190197169A1 (en) * | 2017-12-21 | 2019-06-27 | International Business Machines Corporation | Generating a combined database with data extracted from multiple systems |
-
2019
- 2019-11-27 CN CN201911182873.8A patent/CN111125146B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140258209A1 (en) * | 2013-03-08 | 2014-09-11 | Platfora, Inc. | Systems and Methods for Interest-Driven Distributed Data Server Systems |
| CN103699618A (en) * | 2013-12-16 | 2014-04-02 | 广东威创视讯科技股份有限公司 | Data report generation method and system |
| CN107861933A (en) * | 2017-11-29 | 2018-03-30 | 北京百度网讯科技有限公司 | The method and apparatus for generating O＆M form |
| US20190197169A1 (en) * | 2017-12-21 | 2019-06-27 | International Business Machines Corporation | Generating a combined database with data extracted from multiple systems |
| CN108733799A (en) * | 2018-05-17 | 2018-11-02 | 深圳市买买提信息科技有限公司 | A kind of method for processing report data, device and terminal |
| CN109145051A (en) * | 2018-07-03 | 2019-01-04 | 阿里巴巴集团控股有限公司 | The data summarization method and device and electronic equipment of distributed data base |
Non-Patent Citations (4)
| Title |
|---|
| DANIEL K.C. CHAN: "A Document-driven Approach to Database Report Generation", 《PROCEEDINGS NINTH INTERNATIONAL WORKSHOP ON DATABASE AND EXPERT SYSTEMS APPLICATIONS》, pages 1 - 6 * |
| 冯慧霞、 杨玉华、宋振华: "基于数据集市实现报表自动生成和批量下载", 《信息技术与信息化》, pages 113 - 114 * |
| 宋永利;: "一套智能的报表生成方法及设计原则", no. 11, pages 205 - 206 * |
| 杨廷福，王会梅: "利用VBA编程实现生产日报表的自动生成", 《电脑编程技巧与维护》, pages 23 - 25 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111125146B (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10831562B2 (en) | Method and system for operating a data center by reducing an amount of data to be processed | |
| CN107515878B (en) | Data index management method and device | |
| CN107229619B (en) | Method and device for counting and displaying calling condition of internet service link | |
| CN110058940B (en) | Data processing method and device in multi-thread environment | |
| CN112052138A (en) | Service data quality detection method and device, computer equipment and storage medium | |
| US8849745B2 (en) | Decision support methods and apparatus | |
| CN102054000A (en) | Data querying method, device and system | |
| CN111930770A (en) | Data query method and device and electronic equipment | |
| CN109039817A (en) | A kind of information processing method and device for traffic monitoring | |
| US20150120697A1 (en) | System and method for analysis of a database proxy | |
| EP3803625A1 (en) | Frequent pattern analysis for distributed systems | |
| CN113672375B (en) | Resource allocation prediction method, device, equipment and storage medium | |
| CN110912967A (en) | Service node scheduling method, device, equipment and storage medium | |
| CN113297188B (en) | Data processing method and device | |
| JP2009037369A (en) | Resource assignment method to database server | |
| CN108228355A (en) | Task processing method and device, method for scheduling task and device | |
| CN116700929A (en) | Task batch processing method and system based on artificial intelligence | |
| CN110807058B (en) | Method and system for exporting data | |
| CN111125146A (en) | Report generation method and device and storage medium | |
| CN115525659A (en) | Data query method and device, electronic equipment and storage medium | |
| CN106845926B (en) | Distributed data stream processing method and system for third-party payment supervision system | |
| CN116185578A (en) | Scheduling method of computing task and executing method of computing task | |
| CN113283742A (en) | Task allocation method and device | |
| CN113010310A (en) | Job data processing method and device and server | |
| CN112131257A (en) | Data query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |