CN112347099A - Data processing method and device, computing equipment and computer readable storage medium - Google Patents
Data processing method and device, computing equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN112347099A CN112347099A CN202011163207.2A CN202011163207A CN112347099A CN 112347099 A CN112347099 A CN 112347099A CN 202011163207 A CN202011163207 A CN 202011163207A CN 112347099 A CN112347099 A CN 112347099A
- Authority
- CN
- China
- Prior art keywords
- task
- data
- document
- task document
- message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method, a data processing device, a computing device and a computer readable storage medium, which relate to the technical field of Internet, and the method comprises the following steps: acquiring task document push messages of a plurality of task platforms; the task document pushing message comprises a message generated when task document creating operation and/or task document state changing operation are executed on the basis of each task platform; analyzing the task document pushing message, and acquiring a task document corresponding to the task document pushing message and task data of the task document; and storing the task data into a storage area of the task document in a database, and generating a task document index table based on the task data stored in the database. Based on the method provided by the invention, the task data of the task documents of the plurality of task platforms are gathered through the database, so that the task documents of each task platform are effectively managed in a unified way, and the follow-up task document information query of a user is facilitated.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a data processing method, an apparatus, a computing device, and a computer-readable storage medium.
Background
With the continuous development of computer technology, more and more task platforms are provided. For each task platform, along with the continuous popularization of networks, the amount of generated task documents is increased.
At present, task documents of each task platform are independently managed, and a large amount of redundant data is also stored in a database. And when a user needs to inquire task documents, the user needs to respectively inquire the task documents by each platform, so that the process is complicated, and the user experience is reduced.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data processing method, an apparatus, a computing device, and a computer-readable storage medium, and mainly aim to solve the problem of low user experience caused by that each platform needs to query independently when task data of a task platform is managed independently and data is queried.
According to a first aspect of the embodiments of the present invention, there is provided a data processing method, including:
acquiring task document push messages of a plurality of task platforms; the task document pushing message comprises a message generated when the task document creating operation and/or the task document state changing operation are executed on the basis of each task platform;
analyzing the task document pushing message to obtain a task document corresponding to the task document pushing message and task data of the task document;
storing the task data into a storage area of the task document in a database;
and generating a task document index table based on the task data stored in the database.
In another optional embodiment, the obtaining task document push messages of a plurality of task platforms includes:
and calling a plurality of message monitoring interfaces registered in advance by the task platforms, and acquiring the task document push messages of the task platforms based on the message monitoring interfaces.
In another optional embodiment, after analyzing the task document push message and acquiring the task document corresponding to the task document push message and the task data of the task document, the method further includes:
pre-checking the task data, and judging whether the task data comprises invalid data meeting preset conditions;
and if the task data comprises the invalid data, filtering the invalid data.
In another optional embodiment, after analyzing the task document push message and acquiring the task document corresponding to the task document push message and the task data of the task document, the method further includes:
forwarding the task data to a message queue which is distributed for a task platform to which the task data belongs in advance;
the step of storing the task data into a storage area of the task document in a database comprises the following steps:
and storing the task data into a storage area of the task document in a database through the message queue.
In another optional embodiment, the generating a task document index table based on the task data stored in the database includes:
extracting field data of a plurality of preset fields from the task data of each task document stored in the database;
generating index data of the task document based on the field data of the preset fields;
establishing a task document index table according to the index data of each task document in the database;
the preset fields comprise a task document number, a merchant identifier, a user identifier, a task platform, a task document state, creation time and/or updating time.
In another optional embodiment, the method further comprises:
receiving a first data query request from a client;
identifying a query field in the first data query request; wherein the query field is at least one of the plurality of preset fields;
selecting at least one first target task document in the task document index table according to the query field;
and generating a task document list based on the first target task document and transmitting the task document list to the client.
In another optional embodiment, after the transmitting the list of task documents generated based on the first target task document to the client, the method further includes:
receiving a second data query request of the client aiming at any first target task document;
and acquiring the task data of the first target task form from the database, and transmitting the task data of the first target task form to the client.
In another optional embodiment, the method further comprises:
selecting at least one second target task sheet from the database, and acquiring task data of the second target task sheet;
and summarizing the task data to generate life cycle data of the second target task bill.
In another optional embodiment, the method further comprises:
screening out at least one second target task document associated with the target object from the database;
acquiring life cycle data of each second target task document;
and analyzing the characteristic data of the target object based on the life cycle data of each second target task bill.
According to a second aspect of the embodiments of the present invention, there is provided a data processing apparatus including:
the message acquisition module is used for acquiring task document push messages of a plurality of task platforms; the task document pushing message comprises a message generated when the task document creating operation and/or the task document state changing operation are executed on the basis of each task platform;
the data analysis module is used for analyzing the task document pushing message and acquiring a task document corresponding to the task document pushing message and task data of the task document;
the data storage module is used for storing the task data into a storage area of the task document in a database;
and the index module is used for generating a task document index table based on the task data stored in the database.
In another optional embodiment, the message obtaining module is further configured to:
and calling a plurality of message monitoring interfaces registered in advance by the task platforms, and acquiring the task document push messages of the task platforms based on the message monitoring interfaces.
In another optional embodiment, the apparatus further comprises:
the inspection module is used for pre-inspecting the task data and judging whether the task data comprises invalid data meeting preset conditions;
and if the task data comprises the invalid data, filtering the invalid data.
In another optional embodiment, the apparatus further comprises:
the forwarding module is used for forwarding the task data to a message queue which is distributed for a task platform to which the task data belongs in advance;
the data storage module is also used for storing the task data into a storage area of the task document in a database through the message queue
In another optional embodiment, the apparatus further comprises:
the index module is further used for extracting field data of a plurality of preset fields from the task data of each task document stored in the database;
generating index data of the task document based on the field data of the preset fields;
establishing a task document index table according to the index data of each task document in the database;
the preset fields comprise a task document number, a merchant identifier, a user identifier, a task platform, a task document state, creation time and/or updating time.
In another optional embodiment, the indexing module is further configured to:
receiving a first data query request from a client;
identifying a query field in the first data query request; wherein the query field is at least one of the plurality of preset fields;
selecting at least one first target task document in the task document index table according to the query field;
and generating a task document list based on the first target task document and transmitting the task document list to the client.
In another optional embodiment, the indexing module is further configured to:
receiving a second data query request of the client aiming at any first target task document;
and acquiring the task data of the first target task form from the database, and transmitting the task data of the first target task form to the client.
In another optional embodiment, the data storage module is further configured to select at least one second target task sheet from the database, and obtain task data of the second target task sheet;
and summarizing the task data to generate life cycle data of the second target task bill.
In another optional embodiment, the apparatus further comprises:
the analysis module is used for screening out at least one second target task document associated with the target object from the database;
acquiring life cycle data of each second target task document;
and analyzing the characteristic data of the target object based on the life cycle data of each second target task bill.
According to a third aspect of embodiments of the present invention, there is provided a computing device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of the method of any one of the first aspect when executing the computer program.
According to a fourth aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of any one of the first aspects.
The embodiment of the invention provides a data processing method, a data processing device, a computing device and a computer readable storage medium.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a data processing method according to another embodiment of the present invention;
FIG. 3 is a schematic diagram of a data processing apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a data processing apparatus according to another embodiment of the present invention;
FIG. 5 illustrates a schematic diagram of a computing device provided by an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Fig. 1 shows a schematic flow chart of a data processing method according to an embodiment of the present invention, and as can be seen from fig. 1, the data processing method according to the embodiment of the present invention may at least include steps S102 to S108.
And S102, acquiring task document push messages of a plurality of task platforms.
The task platform in this embodiment is a service platform that can provide different task type services, and corresponds to different fields or different platforms in the same field, for example, the task platform may include a payment platform, a catering platform, a business travel platform, and other e-commerce platforms that provide different types of services. Each task platform can provide different types of task services according to different requirements of users, such as commodity transaction services, travel services, catering services, travel services, payment services and the like. The task document of the task platform may be a task storage record generated by the task platform for any task data, such as an order related record of an e-commerce platform, a payment related record of a payment platform, and the like, which includes but is not limited to a database task data record that can be stored such as order data, commodity record data, payment data, and the like, and besides, data of an operation behavior such as a browsing behavior of the task platform may also be recorded as the task data.
In this embodiment, a push message generated when the task platform detects that the task document is created and/or a message generated when the task platform detects that the task document state is changed can be used as the task document push message. The push message can be an information stream generated by combining data such as task platform type, time, task document number, task document state, operation node of the task document, operation time, state information of the operation node and the like. The push message can be acquired in real time and at a certain time interval, which is not limited in the embodiments of the present invention.
In an optional embodiment of the present invention, before the task document push message of each task platform is obtained, a plurality of message monitoring interfaces registered in advance by the task platform may be called, and the task document push message of each task platform is obtained based on the message monitoring interfaces. The message monitoring interface can be an API interface and can be called by registering a monitoring function. In the embodiment, by adopting a registration message monitoring mode, the push messages of each task platform can be acquired in time, and omission of important push messages is avoided.
And step S104, analyzing the task document pushing message, and acquiring the task document corresponding to the task document pushing message and the task data of the task document.
After the task document push message is obtained, the task document push message can be analyzed to obtain a task document corresponding to the push message and task data related to the task document.
In this embodiment, the task document push message of each task platform may carry task data of the task document, and the task data may include related information of the task document and related information of the task platform. The relevant information of the task document may include task data of a task platform corresponding to the task document, such as a task document number, a task document creation time, a current state of the task document, and the like. In practical application, a task document may have a plurality of operation nodes from creation to completion (taking the task document of an e-commerce platform as an example, the operation nodes of the task document may include nodes for placing orders, paying, delivering, receiving, evaluating, and the like), the related information of the task document may further include the current operation node, operation time, and related operation data at the operation node of the task document, and taking the payment operation node of an order as an example, the related data may include related data such as payment time, payment amount, payment channel, payment method, and the like. The related information of the task platform may include information such as a task platform type, an attribute, and a task platform identifier. Of course, besides the above-mentioned information, other information related to the task document and the task platform may be included, which is not described herein again.
And step S106, storing the task data into a storage area of the task document in the database.
And the task data of the obtained task document can be stored in a database by analyzing. The database in this embodiment is used to store all task data of each task document created by each platform. When the task data is stored, the task data can be stored corresponding to the storage area of the task document to which the task data belongs. Optionally, a data list may be further provided in the database, where the data list may include the number of the task document and a corresponding relationship between the number of the task document and a storage area corresponding to the task document, for example, the corresponding relationship between the number of the task document and a storage address, and after the storage address corresponding to the number of the task document is found in the data list according to the number of the task document, the task data of the task document corresponding to the number of the task document may be obtained.
In an optional embodiment of the present invention, after the task data of the task document is obtained in step S104, the task data may be pre-checked to determine whether the task data includes invalid data meeting a preset condition; and if the task data comprises invalid data, filtering the invalid data, namely deleting the task data, or storing the task data into a specified invalid data storage unit, and regularly cleaning the invalid storage unit. The invalid data of the preset condition in this embodiment may include expired data of the task document, intermediate state data, a type of the task document, and redundant data related to the user and the merchant, and may be set according to different task types and application scenarios, which is not limited in this embodiment of the present invention. It can be understood that the data stored in the database is mirror image data of the task document corresponding to each task platform, and the mirror image data is related to the task document after redundant data is filtered compared with original task data of the task document.
In addition to the above description, when determining whether the task data includes invalid data meeting the preset condition, the determination may be performed based on other push messages received in the same time period. For example, if a plurality of pieces of push information are continuously obtained for the same task document, and different state information may be recorded in the plurality of pieces of push information for the same operation node of the task document, the state information in the push information with relatively long obtaining time may be used as invalid data. For example, the operation node displaying the task document in the push message 1 is to be paid, and the state of the operation node is that the payment is unsuccessful; and if the operation node displaying the task document in the push message 2 is to-be-paid and the operation node is in a successful payment state, the push message 1 can be judged to be invalid data and filtered. Based on the method provided by the embodiment of the invention, the effectiveness of the data stored in the database can be controlled by pre-checking the task data to filter invalid data, so that the storage space of the database is reasonably utilized.
In another optional embodiment of the present invention, after the task document corresponding to the push message and the task data of the task document are acquired in step S104, the task data may be forwarded to a message queue allocated to a task platform to which the task data belongs in advance, and then the task data is stored in a storage area of the task document in the database through the message queue.
A message queue is a container that holds messages during their transmission. The primary purpose of the queues is to provide routing and to guarantee delivery of the messages. In the embodiment of the invention, the task data is stored in the preset message storage queue, so that the task data in the message queue can be orderly stored in the database while the task data is ensured to be stored, and the data processing speed is improved.
Generally speaking, the task data forwarded to the message queue has corresponding time parameters, which may include a first time parameter of a push message to which the task data belongs, a second time parameter recorded when the push message is generated by the task platform, and a third time parameter of a state machine of the task platform, and when data processing is performed through the message queue, the at least two time parameters are combined as corresponding time parameters in the task data, and the data is sequentially processed according to a time sequence of the task data.
In an optional embodiment of the present invention, multiple groups of message queues may be preset, and when the task data is forwarded to the message queues, the task data may be distributed according to the data amount in each group of message queues, or may be distributed according to a task platform of the task data. For example, if the data size of the task platform 1 is large, the relevant task data of the task platform 1 may be forwarded to the specified message queue 1. Assuming that the data amount of the task platform 2 and the task platform 3 is small, the task data related to the task platform 2 and the task platform 3 can be forwarded to the message queue 2 at the same time. In this embodiment, the corresponding relationship between the task platforms and the message queues may be pre-established, so that after receiving a push message of any task platform, the task data of the task platform may be directly forwarded to the corresponding message queue, thereby reducing time consumption.
And step S108, generating a task document index table based on the task data stored in the database.
By the introduction, the database may store task data of each task document created by each platform. In the optional embodiment of the invention, a task document index table can be established to reduce the data access pressure on the database caused by the task library during each data query; the task document index table comprises field data of at least one preset field of each task document, and the preset field can be a predetermined key field of the task document, such as document number, document state and the like. Optionally, the process of establishing the task document index table may be as follows:
and S1-1, extracting field data of a plurality of preset fields from the task data of each task document stored in the database. The preset fields can comprise list necessary fields such as task document numbers, merchant identifications, user identifications, task platforms, task document states, creation time and updating time. For each task document, the field data corresponding to each preset field may be extracted from the database.
And S1-2, generating index data of the task document based on the field data of the plurality of preset fields. After the field data of each preset field of each task document is obtained, the field data can be integrated to obtain the index data corresponding to each task document.
And S1-3, establishing a task document index table according to the index data of each task document in the database.
After the index data of each task document is obtained, a task document index table containing field data of the preset field of each task document can be established. According to the method provided by the embodiment of the invention, the key information management can be carried out on each task document of each platform based on the task document index table by establishing the task document index table. Optionally, the data stored in the database can be indexed by the search engine based on the task document index table, the data size is small because only necessary task document information is stored in the task document index table, the control part divides words by the search engine index field, and the expansion rate is lower than that of the database index.
Further, the method provided by the embodiment of the present invention may also return different data according to different data query requests. The process may be as follows:
s2-1, a first data query request from a client is received. The client may be a client corresponding to the user or a client corresponding to the merchant. The first data query request may be a task document query request for any merchant or user.
S2-2, identifying a query field in the first data query request. Wherein, the query field is at least one of a plurality of preset fields. The query field may be an identification of a target object that initiated the data query request. Assuming that the first data query request is a data query request from a client of a user, an identification of the user may be obtained as a query field.
And S2-3, selecting at least one first target task sheet in the task sheet index table according to the query field.
Because the index table is established based on the preset fields and the field data of the preset fields, after the query fields are identified, all task documents with the query fields can be screened out from the index table.
And S2-4, generating a task document list based on the first target task document and transmitting the task document list to the client.
In this embodiment, the database stores the task data of the task documents of each task platform, and the task document list generated finally is the task document list of all the task platforms obtained based on the query field. The task document list can be classified and displayed after task tags are generated by the task platform corresponding to each first target task document.
Taking the first data query request as an example from the client of the user, the task document list finally returned to the client is a list formed by the relevant task documents of the user on all task platforms. Each task platform can have a respective task tag, and the task document list can be classified and displayed according to the task tags, so that the user can check the task document list conveniently. According to the method provided by the embodiment of the invention, the task document index table is established, so that when a target object inquires brief information of the task document, the target object can directly inquire the brief information of the task document based on the task document index table, the task document index table only stores key information of the task document, the data size is small, and the field is used as an inquiry basis to inquire in the task document index list, so that the expansion rate of the list index is lower than that of a database index. In addition, the database in the embodiment stores the task documents of each task platform, task data of a plurality of task platforms are gathered, and the task documents of the plurality of task platforms can be searched only by executing one query operation, so that the user experience can be effectively improved.
In practical application, related data in the task document index table may have a data loss condition in corresponding data, in the embodiment of the invention, the data in the task document index table can be regularly checked, and when data of any task document is lost, the data of the corresponding task document can be obtained based on the database again, so that the task document index table can be updated to a latest state in time.
In an alternative embodiment of the present invention, after step S2-4, the method may further include:
step S2-5, receiving a second data query request of the client aiming at any first target task document;
and step S2-6, acquiring the task data of the first target task sheet from the database, and transmitting the task data of the first target task sheet to the client.
In this embodiment, it is assumed that the client needs to obtain detailed information of any task document related to the client, and the client can call the task document based on the database. In the above embodiment, the database may have a data list, and when the detailed information of any task document is acquired, the task document number of the inquired task document may be identified based on the second data inquiry request, and then the detailed task data of the task document may be inquired in the database according to the task document number.
In the foregoing, the task data of each task document stored in the database may include related information of the task document and related information of the task platform, and the related information of the task document includes related operation data and time and other information corresponding to each operation node. Optionally, at least one second target task sheet is selected from the database, and task data of the second target task sheet is obtained; and summarizing the task data to generate life cycle data of a second target task document.
In this embodiment, when the second target task document is selected, the second target task document may be selected according to the state of each task document in the database, that is, the task document whose task document state is the end state is selected as the second target task document, and the task document state of one task document is the end state, that is, all the operation nodes of the task document are executed and completed, for example, the task document of one e-commerce platform is executed and completed from the start of creation to the completion of evaluation. After the second target task document is selected, all task data of the second target task document can be obtained and summarized. The task data stored in the second target task document each time also carries time information, so that when the task data of the second target task document is summarized, all the task data can be sequentially summarized by taking the time information as a basis, and further the life cycle data of the second target task document is generated.
That is, a life cycle axis of the second target task document can be established based on time, and the operation time, the task document state, the related task data and the like of the task document at each operation node are recorded from the creation of the task document to the completion of the task document.
Further, in an optional embodiment of the present invention, at least one second target task document associated with the target object may be screened out from the database; acquiring life cycle data of each second target task document; and analyzing the characteristic data of the target object based on the life cycle data of each second target task document. The target object may be any user, or a business, or may be a task platform. If the target object is a user, the task document of the user is established by the user on one task platform or a plurality of task platforms. Similarly, if the target object is a merchant, the merchant's task document created by the merchant on one task platform or a plurality of task platforms can be screened out.
And after the second target task documents are screened out, life cycle data of each second target task document can be obtained, and then the characteristic data of the target object is analyzed according to the life cycle data. For example, when the target object is a user, the characteristics of the target object, such as preference setting and the like, are analyzed through comparison data of the target object on any task platform, commodity adding and deleting data, purchased commodity characteristic data, key information of each commodity, price intervals of different types of articles in each task document, evaluation data of the task document and the like, so that commodities meeting the requirements of the target object can be subsequently recommended to the target object. In addition, for the same type of commodities, different task documents may be in a transaction ending state and a transaction success state respectively, so that user behavior characteristic analysis can be performed by comparing the two groups of task documents. For the merchant, audience groups and transaction characteristics of different commodities can be analyzed through the transaction amount and transaction time of each commodity.
According to the method provided by the embodiment of the invention, the task data of the task document is summarized to obtain the life cycle data, so that the relevant data of the task document can be effectively managed, and meanwhile, the characteristic data of different objects can be analyzed based on the life cycle data, thereby providing guidance information for the target object, such as merchant commodity storage, user information recommendation and the like.
An embodiment of the present invention provides a data processing method, and as shown in fig. 2, the data processing method provided in this embodiment may include:
step S202, registering message monitoring on a plurality of task platforms to obtain task document pushing messages of the plurality of task platforms; each task platform can respectively correspond to different fields or the same field; for example, a task document push message 1 when a task document 1 pushed by a catering platform is created and a task document push message 2 when the state of a task document 2 pushed by a trip platform is updated can be obtained.
Step S204, the task document push message is analyzed, and a task document corresponding to the task document push message and task data of the task document are obtained.
For example, the task document push message 1 is parsed, and a task document number, a task platform to which the task document push message belongs, a user name, a user identifier, other related information of the user, a merchant name, a merchant identifier, merchant shop information, other related information of the merchant, purchased article information, task document creation time, a task channel (for example, through a client or a webpage), and the like of the task document 1 are obtained.
And analyzing the task document pushing message 1 to obtain the task document number, the affiliated task platform, the user information, the merchant information, the purchased article information, the current task document state and the state updating time of the task document 2. And if the current task document state is a paid state, acquiring a payment channel, a payment mode, payment time, payment amount and the like.
Step S206, pre-checking the task data, and determining whether the task data includes invalid data. Taking the task data of the analysis task document push message 1 as an example, the detailed information of the user and the merchant included in the task document push message may be invalid data, such as other related information of the user, merchant store information, and other related information of the merchant, which may be invalid data.
In step S208, invalid data is filtered.
Step S210, forwarding the task data to a pre-distributed message queue; for example, because the amount of the task documents of the catering platform and the trip platform is relatively large, the message queue 1 can be allocated to the catering platform in advance, and the message queue 2 can be allocated to the trip platform. The task data of the push message 1 is forwarded to the message queue 1, and the task data of the push message 2 is forwarded to the message queue 2;
and step S212, storing the task data into a storage area of the task document in the database through the message queue. Namely, the task data of the task document 1 is stored in the storage area corresponding to the task document 1, and the task data of the task document 2 is stored in the storage area corresponding to the task document 2.
And step S214, generating a task document index table based on the task data of each task document in the database. Specifically, field data corresponding to fields such as a task document number, a merchant identifier, a user identifier, a task platform, a task document state, creation time and update time can be extracted based on task data of each task document, and then a task document index table is generated in a gathering manner.
Step S216, receiving a first query request of a client for a user A, searching all task documents of the user A in a task document index table, and returning a task document list to the client;
step S218, receiving a detailed task data viewing request of the client for any task document in the task document list, and obtaining corresponding task data in a storage area corresponding to the database according to the serial number of the task document and returning the corresponding task data to the client.
The embodiment of the invention provides a data processing method, which can obtain task document push messages of a plurality of platforms in a message monitoring mode and can simultaneously ensure second-level availability of task document information of a merchant end and a user end of a task platform. Further, the task data corresponding to each task document is analyzed from the task document push message, invalid data in the task data are filtered and then stored in the storage area corresponding to the task document, and the occupation of redundant data on the storage space of the database can be reduced while data storage is achieved.
Further, the database in this embodiment stores task documents of each task platform, aggregates task data of a plurality of task platforms, and establishes an index list to use a field as a query basis to query key information of the task documents. Compared with information query directly based on a database, the method can effectively save data space, simultaneously realize decoupling of task document list query and the database, and reduce pressure of the database. In addition, in the method provided by the embodiment of the invention, the joint query of the task documents of the plurality of task platforms can be realized only by executing one query operation, so that the user or a merchant can conveniently check and manage the task document data of the plurality of task platforms, and the user experience is further improved.
Based on the same inventive concept, an embodiment of the present invention further provides a data processing apparatus, as shown in fig. 3, including:
the message acquiring module 310 is configured to acquire task document push messages of a plurality of task platforms; the task document pushing message comprises a message generated when task document creating operation and/or task document state changing operation are executed on the basis of each task platform;
the data analysis module 320 is configured to analyze the task document push message, and obtain a task document corresponding to the task document push message and task data of the task document;
the data storage module 330 is used for storing the task data into a storage area of the task document in the database;
and the indexing module 340 is configured to generate a task document index table based on the task data stored in the database.
In an optional embodiment of the present invention, the message obtaining module is further configured to:
and calling a plurality of message monitoring interfaces registered in advance by the task platforms, and acquiring the task document push messages of the task platforms based on the message monitoring interfaces.
In an alternative embodiment of the present invention, as shown in fig. 4, the apparatus may further include:
the checking module 350 is configured to perform pre-checking on the task data, and determine whether the task data includes invalid data meeting a preset condition;
and if the task data comprises invalid data, filtering the invalid data.
In an alternative embodiment of the present invention, as shown in fig. 4, the apparatus may further include:
the forwarding module 360 is configured to forward the task data to a message queue that is pre-allocated to a task platform to which the task data belongs;
the data storage module 330 is further used for storing the task data into the storage area of the task document in the database via the message queue
In an optional embodiment of the present invention, as shown in fig. 4, the indexing module 340 may be further configured to extract field data of a plurality of preset fields from the task data of each task document stored in the database;
generating index data of the task document based on field data of a plurality of preset fields;
establishing a task document index table according to the index data of each task document in the database;
the preset fields comprise a task document number, a merchant identifier, a user identifier, a task platform, a task document state, creation time and/or updating time.
In an optional embodiment of the present invention, the indexing module 340 may further be configured to:
receiving a first data query request from a client;
identifying a query field in the first data query request; wherein, the query field is at least one of a plurality of preset fields;
selecting at least one first target task document in a task document index table according to the query field;
and generating a task document list based on the first target task document and transmitting the task document list to the client.
In an optional embodiment of the present invention, the indexing module 340 may further be configured to:
receiving a second data query request of the client aiming at any first target task bill;
and acquiring the task data of the first target task document from the database, and transmitting the task data of the first target task document to the client.
In an optional embodiment of the present invention, the data storage module 330 may further be configured to:
selecting at least one second target task document from the database, and acquiring task data of the second target task document;
and summarizing the task data to generate life cycle data of a second target task document.
In an alternative embodiment of the present invention, as shown in fig. 4, the apparatus may further include:
an analysis module 370 for screening out at least one second target task document associated with the target object in the database;
acquiring life cycle data of each second target task document;
and analyzing the characteristic data of the target object based on the life cycle data of each second target task document.
It should be noted that other corresponding descriptions of the functional units related to the data processing apparatus provided in the embodiment of the present invention may refer to the corresponding descriptions in fig. 1 to fig. 2, and are not described herein again.
In an exemplary embodiment, referring to fig. 5, a computing device is further provided, where the computing device includes a communication bus, a processor, a memory, a communication interface, an input/output interface, and a display device, and the functional units may communicate with each other through the bus. The memory stores computer programs, and the processor is used for executing the programs stored in the memory and executing the steps of the data processing method in the embodiment.
In an alternative embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the data processing method.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by hardware, and also by software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the implementation scenarios of the present application.
Those skilled in the art will appreciate that the figures are merely schematic representations of one preferred implementation scenario and that the blocks or flow diagrams in the figures are not necessarily required to practice the present application.
Those skilled in the art will appreciate that the modules in the devices in the implementation scenario may be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the present implementation scenario with corresponding changes. The modules of the implementation scenario may be combined into one module, or may be further split into a plurality of sub-modules.
The above application serial numbers are for description purposes only and do not represent the superiority or inferiority of the implementation scenarios.
The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present application.
Claims (10)
1. A data processing method, comprising:
acquiring task document push messages of a plurality of task platforms; the task document pushing message comprises a message generated when the task document creating operation and/or the task document state changing operation are executed on the basis of each task platform;
analyzing the task document pushing message to obtain a task document corresponding to the task document pushing message and task data of the task document;
storing the task data into a storage area of the task document in a database;
and generating a task document index table based on the task data stored in the database.
2. The method of claim 1, wherein obtaining task document push messages for a plurality of task platforms comprises:
and calling a plurality of message monitoring interfaces registered in advance by the task platforms, and acquiring the task document push messages of the task platforms based on the message monitoring interfaces.
3. The method according to claim 1, wherein after analyzing the task document push message and acquiring the task document corresponding to the task document push message and the task data of the task document, the method further comprises:
pre-checking the task data, and judging whether the task data comprises invalid data meeting preset conditions;
and if the task data comprises the invalid data, filtering the invalid data.
4. The method according to claim 1, wherein after analyzing the task document push message and acquiring the task document corresponding to the task document push message and the task data of the task document, the method further comprises:
forwarding the task data to a message queue which is distributed for a task platform to which the task data belongs in advance;
the step of storing the task data into a storage area of the task document in a database comprises the following steps:
and storing the task data into a storage area of the task document in a database through the message queue.
5. The method of claim 1, wherein generating a task document index table based on task data stored in the database comprises:
extracting field data of a plurality of preset fields from the task data of each task document stored in the database;
generating index data of the task document based on the field data of the preset fields;
establishing a task document index table according to the index data of each task document in the database;
the preset fields comprise a task document number, a merchant identifier, a user identifier, a task platform, a task document state, creation time and/or updating time.
6. The method of claim 5, further comprising:
receiving a first data query request from a client;
identifying a query field in the first data query request; wherein the query field is at least one of the plurality of preset fields;
selecting at least one first target task document in the task document index table according to the query field;
and generating a task document list based on the first target task document and transmitting the task document list to the client.
7. The method of claim 6, wherein after the generating a list of task documents based on the first target task document is transmitted to the client, further comprising:
receiving a second data query request of the client aiming at any first target task document;
and acquiring the task data of the first target task form from the database, and transmitting the task data of the first target task form to the client.
8. A data processing apparatus, comprising:
the message acquisition module is used for acquiring task document push messages of a plurality of task platforms; the task document pushing message comprises a message generated when the task document creating operation and/or the task document state changing operation are executed on the basis of each task platform;
the data analysis module is used for analyzing the task document pushing message and acquiring a task document corresponding to the task document pushing message and task data of the task document;
the data storage module is used for storing the task data into a storage area of the task document in a database;
and the index module is used for generating a task document index table based on the task data stored in the database.
9. A computing device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011163207.2A CN112347099A (en) | 2020-10-27 | 2020-10-27 | Data processing method and device, computing equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011163207.2A CN112347099A (en) | 2020-10-27 | 2020-10-27 | Data processing method and device, computing equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112347099A true CN112347099A (en) | 2021-02-09 |
Family
ID=74358662
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011163207.2A Pending CN112347099A (en) | 2020-10-27 | 2020-10-27 | Data processing method and device, computing equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347099A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568907A (en) * | 2021-07-14 | 2021-10-29 | 彩讯科技股份有限公司 | Method, device, server and storage medium for processing station information message |
| CN117635081A (en) * | 2024-01-25 | 2024-03-01 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8819145B1 (en) * | 2007-10-08 | 2014-08-26 | Sprint Communications Company L.P. | System and method for message presentation consolidation |
| CN108009028A (en) * | 2017-11-29 | 2018-05-08 | 中国平安人寿保险股份有限公司 | Message treatment method, device, equipment and computer-readable recording medium |
| CN108804697A (en) * | 2018-06-15 | 2018-11-13 | 中国平安人寿保险股份有限公司 | Method of data synchronization, device, computer equipment based on Spark and storage medium |
| CN110990396A (en) * | 2019-10-12 | 2020-04-10 | 贝壳技术有限公司 | Data storage method, device and storage medium |
| CN111198892A (en) * | 2019-12-24 | 2020-05-26 | 泰康保险集团股份有限公司 | Information processing method, information processing device, electronic equipment and storage medium |
| CN111242595A (en) * | 2019-12-31 | 2020-06-05 | 航天信息股份有限公司企业服务分公司 | Information exchange method and device between ERP platform and payment platform |
| CN111652681A (en) * | 2020-05-29 | 2020-09-11 | 平安医疗健康管理股份有限公司 | Receipt processing method, server and computer readable storage medium |
| CN111709817A (en) * | 2020-06-23 | 2020-09-25 | 腾讯科技（深圳）有限公司 | Order data processing method and device, computer readable medium and electronic equipment |
-
2020
- 2020-10-27 CN CN202011163207.2A patent/CN112347099A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8819145B1 (en) * | 2007-10-08 | 2014-08-26 | Sprint Communications Company L.P. | System and method for message presentation consolidation |
| CN108009028A (en) * | 2017-11-29 | 2018-05-08 | 中国平安人寿保险股份有限公司 | Message treatment method, device, equipment and computer-readable recording medium |
| CN108804697A (en) * | 2018-06-15 | 2018-11-13 | 中国平安人寿保险股份有限公司 | Method of data synchronization, device, computer equipment based on Spark and storage medium |
| CN110990396A (en) * | 2019-10-12 | 2020-04-10 | 贝壳技术有限公司 | Data storage method, device and storage medium |
| CN111198892A (en) * | 2019-12-24 | 2020-05-26 | 泰康保险集团股份有限公司 | Information processing method, information processing device, electronic equipment and storage medium |
| CN111242595A (en) * | 2019-12-31 | 2020-06-05 | 航天信息股份有限公司企业服务分公司 | Information exchange method and device between ERP platform and payment platform |
| CN111652681A (en) * | 2020-05-29 | 2020-09-11 | 平安医疗健康管理股份有限公司 | Receipt processing method, server and computer readable storage medium |
| CN111709817A (en) * | 2020-06-23 | 2020-09-25 | 腾讯科技（深圳）有限公司 | Order data processing method and device, computer readable medium and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568907A (en) * | 2021-07-14 | 2021-10-29 | 彩讯科技股份有限公司 | Method, device, server and storage medium for processing station information message |
| CN117635081A (en) * | 2024-01-25 | 2024-03-01 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN117635081B (en) * | 2024-01-25 | 2024-04-16 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107193894B (en) | Data processing method, individual identification method and related device | |
| CN106844372B (en) | Logistics information query method and device | |
| CN110020086B (en) | User portrait query method and device | |
| CN111046237B (en) | User behavior data processing method and device, electronic equipment and readable medium | |
| CN111459985A (en) | Identification information processing method and device | |
| CN109213758B (en) | Data access method, device, equipment and computer readable storage medium | |
| EP3076359A1 (en) | Implementing retail customer analytics data model in a distributed computing environment | |
| CN103106585A (en) | Real-time duplication eliminating method and device of product information | |
| CN107291779B (en) | Cache data management method and device | |
| CN103207882A (en) | Shop visiting data processing method and system | |
| CN105760380A (en) | Database query method, device and system | |
| CN112347099A (en) | Data processing method and device, computing equipment and computer readable storage medium | |
| CN112506925A (en) | Data retrieval system and method based on block chain | |
| US20160092960A1 (en) | Product recommendations over multiple stores | |
| CN106649602A (en) | Way, device and server of processing business object data | |
| CN108491408B (en) | Activity information processing method and device, electronic equipment and storage medium | |
| CN111177481B (en) | User identifier mapping method and device | |
| CN114064705A (en) | User information fusion method, terminal, storage medium and system under multilayer association | |
| CN108416645B (en) | Recommendation method, device, storage medium and equipment for user | |
| CN107545039B (en) | Keyword index acquisition method and device, computer equipment and storage medium | |
| CN109165217A (en) | A kind of high-efficiency storage method of time series data | |
| CN112052259A (en) | Data processing method, device, equipment and computer storage medium | |
| CN110008243A (en) | A kind of tables of data processing method and processing device | |
| CN107679198B (en) | Information query method and device | |
| CN108345600B (en) | Management of search application, data search method and device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |