CA3128347A1

CA3128347A1 - Multi-data-stream processing method, apparutus, computer device and storage medium

Info

Publication number: CA3128347A1
Application number: CA3128347A
Authority: CA
Inventors: Jinying WU; Dawei Yang; Ruimin Guo; Jinjin QIAN; Xiaobo SI
Original assignee: 10353744 Canada Ltd
Current assignee: 10353744 Canada Ltd
Priority date: 2020-08-17
Filing date: 2021-08-16
Publication date: 2022-02-17
Also published as: CN112115136B; CN112115136A

Abstract

The present invention discloses to a multi-data-stream processing method, apparatus, computer device, and storage medium. The method comprises: continuously receiving first streaming data and second streaming data from streaming processing platform;
after receiving any first streaming data that has correspondingly second streaming data, if all the second streaming data are received, performing association processing of the first streaming data with all the second streaming data, when the obtaining associated data meets preset conditions, storing the first streaming data into a database; after receiving any second streaming data, if the target first streaming data corresponding to the second streaming data and all the second streaming data are received, then performing association processing of the target first streaming data with all the second streaming data, when the obtaining associated data meets preset conditions, storing the target first streaming data into a database that can achieve multi-data-stream processing in real time.

Description

MULTI-DATA-STREAM PROCESSING METHOD, APPARUTUS, COMPUTER
DEVICE AND STORAGE MEDIUM
Field [0001] The present disclosure relates to the field of big data real-time processing, particularly to a multi-data-stream processing method, an apparatus, a computer device, and a storage medium.
Background

[0002] In the price control business system, the price is divided into two kinds of order data:
sales amount and payment amount. For the price control, if only according to the sales amount control, the control effect will be low; if only according to actual amount control, then the control effect will be strict. To achieve the expected effect, it is necessary to use the actual payment amount of the order and the coupon information to calculate the actual payment amount for control.

[0003] Currently most general way is storing the multiple real-time data streams to a relational database or HDFS (Hadoop Distributed File System) first, then providing business with offline query function, obtaining the result set that needs to be processed by association query from plural tables in the relational database or HDFS, then passing the result set to the business system for processing.

[0004] However, the method based on offline technology for results query has high delay, so that the above method cannot guarantee the real-time data. Due to the uncertainty of data entry time of the multiple real-time data steams after processing, it is easy to lead data loss during association query, so the above method cannot guarantee the data reliability.
In addition, the multiple interactive real-time data streams may not be worth saving or are not necessarily redundant in some types of databases. Therefore, the above method may occupy too many storage resources and resulting in low utilization of storage resources in the system.

Date Recue/Date Received 2021-10-18 Invention Content

[0005] Based on the current technical problems, the present invention provides a multi-data-stream processing method, an apparatus, a computer device and a storage medium, the implementations of the present invention can perform multiple real-time data stream processing that obtaining more accurate data after processing and reducing the occupancies of storage resources.

[0006] The present invention provides a multi-data-stream processing method according to the first aspect, in an implementation, the method includes:

[0007] Continuously receiving first streaming data and second streaming data from a streaming processing platform, any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data;

[0008] After receiving any first streaming data, wherein the first streaming data has the correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;

[0009] After receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database Date Recue/Date Received 2021-10-18

[0010] In an implementation, before the step of performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, comprising

[0011] Obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;

[0012] Determining a second data identification corresponding to the first data identification, the quantity of the second data identification corresponding to the first data identification is N, N is a positive integer greater than 0;

[0013] Generating a first data index according to the first data identification;

[0014] Querying the second data identification corresponding to the first data index in the data storage middleware, the quantity of the second data identification corresponding to the first data index is M, M is a positive integer greater than 0;

[0015] If the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and

[0016] If the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, detelliiining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

[0017] In an implementation, after the step of determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data Date Recue/Date Received 2021-10-18 has the correspondingly second streaming data, comprising:

[0018] Determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;

[0019] Generating a second data index according to the first data index;

[0020] Storing the second data index and the complement data into the data storage middleware.

[0021] The multi-data-stream processing method according to claim 1, before the step of performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, comprising:

[0022] Determining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;

[0023] Generating a second data index according to the first data identification corresponding to the second data identification;

[0024] Querying the data storage middleware according to the second data index;

[0025] If the second data identification corresponding to the second data index exists in the data storage middleware, determining of the target first streaming data corresponding to any of the second streaming data is received;

[0026] If the second data identification corresponding to the second data index does not exist in the data storage middleware, determining of the target first streaming data corresponding to any second streaming data is not received.

Date Recue/Date Received 2021-10-18

[0027] In an implementation, the method also includes:

[0028] If the second data identification corresponding to the second data index exists in the data storage middleware, determining whether the second data identification is the same as the second data identification corresponding to the second data index;

[0029] If the second data identification and the second data identification corresponding to the second data index are the same, determining of all the second streaming data corresponding to the target first streaming data are received;

[0030] If the second data identification and the second data identification corresponding to the second data index are not the same, determining of all the second streaming data corresponding to the target first streaming data are not all received;

[0031] Preferably, the method also comprises:

[0032] If the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification;

[0033] Storing the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

[0034] In an implementation, after the step of determining of all the second streaming data corresponding to the target first streaming data are not all received, comprising:

[0035] According to the second data identification, updating the second data identification corresponding to the second data identification in the data storage middleware.
Date Recue/Date Received 2021-10-18

[0036] In an implementation, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, comprising:

[0037] Determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;

[0038] Determining a second amount corresponding to each second streaming data;

[0039] Obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, the third amount is the associated data after performing the association processing;

[0040] In an implementation, the steps of performing association processing of the target first streaming data with all the corresponding second streaming data, comprising:

[0041] Determining the first amount corresponding to the target first streaming data;

[0042] Determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data;

[0043] Obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

[0044] The present invention provides an apparatus for the multi-data-streaming processing, wherein the apparatus comprises:

Date Recue/Date Received 2021-10-18

[0045] A streaming data receiving module configured to continuously receive the first streaming data and the second streaming data from a streaming processing platform, any first streaming data corresponds to zero or at least one second streaming data, any second streaming data corresponds to one first streaming data;

[0046] A first data processing module configured to after receiving any receiving any first streaming data, wherein the first streaming data has correspondingly second streaming data, if determining of all the corresponding second streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;

[0047] A second data processing module configured to after receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database.

[0048] According to the third aspect, the present invention provides a computer device including a memory, a processor and a computer program stored in the memory and ran on the processor configured to achieve the steps of any above-mentioned methods when the processor executes the computer program.

[0049] According to the fourth aspect, the present invention provides a computer readable storage medium stored with a computer program configured to achieve the steps of any above-mentioned methods when processor executes the computer program.

Date Recue/Date Received 2021-10-18

[0050] In the implementation of present invention, the present server continuously receives first streaming data and second streaming data from a streaming processing platform, any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data; after receiving any first streaming data, wherein the first streaming data has the correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data; and after receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database. The implementation of present invention introduces a data storage middleware that achieves the multiple real-time data stream, which means dynamically merging the first data steam and the second data stream, comparing with the current technology which firstly storing the first data steam and the second data stream to the database and then obtaining the results by performing processing of the first data stream and the second data stream in the database, the present invention method can perform more real-time processing on multiple real-time data stream, and the obtaining data results can be more accurate that can reduce the storage resources.
Drawing Description

[0051] Figure 1 is an application environment diagram of a multi-data-stream processing method in an implementation;

Date Recue/Date Received 2021-10-18

[0052] Figure 2 is a process diagram of a multi-data-stream processing method in an implementation;

[0053] Figure 3 is a structural diagram of a multi-data-stream processing apparatus in an implementation;

[0054] Figure 4 is an internal structure of a computer device in an implementation.
Specific implementation methods

[0055] In order to make application purposes, technical solutions, and advantages clearer, the present disclosure is further explained in detail with a particular embodiment thereof, and with reference to the drawings. It shall be appreciated that these descriptions are only intended to be illustrative, but not to limit the scope of the disclosure thereto.

[0056] An implementation of the present invention as shown in Figure 1 is the application environment of the multi-data-stream processing method.

[0057] Wherein, server 10 is a stream processing platform that possibly can be implemented by Kafka stream processing platform. Server 20 is docking server to the server 10, the server 20 will access the first data stream and the second data stream from the server 10, the data of the first data stream is the first streaming data, the data of the first data steam is the second streaming data, the first streaming data includes at least the first data identification, the second streaming data includes at least the second data identification. In some application scenarios, there is correspondence relations between the first streaming data from the first data stream and one or plural second streaming data from the second data stream. For example, in an order's price control scenario, the first data stream is order data, the second data stream is promotional data, in this scenario, user can purchase many kinds of products in one order, this time the order corresponds to a big order number, and each product in the order corresponds to a small order Date Recue/Date Received 2021-10-18 number (order line number), when user purchases the products, user can either choose to not use coupon or choose to use coupon (is able to use at least one coupon), wherein, the promotion data can include coupon number at least, which refers to the only data identification of the coupon used by user, if user does not use the coupon when purchasing products, then there is no need to perform processing on the order data corresponding to the products, but filtering out this order data, if user uses the coupon to purchase products, then the order data of the products includes order line number, coupon number, and coupon's amount corresponding to the coupon's number.

[0058] The server 20 performs association processing of any first streaming data with all the second streaming data corresponding to the first streaming data after receiving any first streaming data and all the second streaming data corresponding to the first streaming data, then obtaining the associated data, when the associated data meets preset conditions, storing any first streaming data into a database 40. Wherein the server 20 will firstly store the first streaming data and the second streaming data sent by the server 10 to data storage middleware 30, then performing association processing of the first streaming data and all the second streaming data correspondingly to the first streaming data, obtaining the associated results.

[0059] Wherein, the server 10 and the server 20 can be implemented as independent server or as a cluster server composed of plural servers. The server 20, the data storage middleware 30 and the database 40 can be deployed on the same server or a cluster of servers, they can also respectively be deployed on an independent server or a cluster of servers. As an example, the Redis database can be used as data storage middleware 30, the MYSQL database can be used as database 40.

[0060] As shown in Figure 2, the implementation of the present invention provides a multi-data-stream processing method, the method includes following steps:
Date Recue/Date Received 2021-10-18

[0061] S110, continuously receiving first streaming data and second streaming data from a streaming processing platform, any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data.

[0062] In the present implementation, the server 20 (hereinafter referred as the server) will access the first data stream and the second data stream of the streaming processing platform, then continuously receiving first streaming data and second streaming data from a streaming processing platform. Any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data. Wherein, the first streaming data includes the first data identification, the second streaming data includes the second data identification, in a possible implementation, for the first streaming data which has correspondingly second data identification, the first streaming data not only has the first data identification but also has all second data identifications. In different application scenarios, the first data identification and the second identification can be different data, such as in the order price control scenario, the first data identification can be order number or the order line number, the second data identification can be coupon number.

[0063] For any first streaming data from the first data stream, if existing the second streaming data corresponding to the first streaming data in the second data stream, for the any first streaming data and the second streaming data corresponding to the first streaming data, their order of arriving at server is random. Which means the first streaming data may arrive at the server earlier than the second streaming data, and also may arrive at the server later that all the other second streaming data; if the first streaming data has correspondingly two or more second streaming data, the first streaming data may arrive at the server later than parts of its correspondingly second streaming data, but arriving at the server earlier that the rest parts of its correspondingly second streaming data, for example, the second streaming data are B and C corresponding to the first streaming data A, in A, B, C, B arrives at the server earliest, then Date Recue/Date Received 2021-10-18 A arrives at the server, and C arrives at the server at last. But the server will perfoim association processing only after any first streaming data and all its correspondingly second streaming data arrive at the server.

[0064] Considering the arriving order of the first streaming data and its correspondingly second streaming data to the server is hard to determine, and performing association processing only when both of the first streaming data and its correspondingly second streaming data arrive at the server, therefore, the processing logics are setting respectively to the first streaming data and the second streaming data, to achieve the association processing of both of them, which can improve the real-time performance of the first data stream and the second data stream.

[0065] S120, after receiving any first streaming data, wherein the first streaming data has the correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data.

[0066] In the present implementation, after the server receives the first streaming data, judging whether the first streaming data has correspondingly second streaming data. In a possible implementation, judging method can be judging whether existing the second data identification except the first data identification in the first streaming data.

[0067] The procedure of the server judges whether all the second streaming data corresponding to the first streaming data are received, as following:

[0068] Obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;

Date Recue/Date Received 2021-10-18

[0069] Determining a second data identification corresponding to the first data identification, the quantity of the second data identification corresponding to the first data identification is N, N is a positive integer greater than 0;

[0070] Generating a first data index according to the first data identification;

[0071] Querying the second data identification corresponding to the first data index in the data storage middleware, the quantity of the second data identification corresponding to the first data index is M, M is a positive integer greater than 0;

[0072] If the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, detellnining of all the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data;

[0073] If the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, detellnining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

[0074] After determining of the received first streaming data has the correspondingly second streaming data, if the server determines all the second streaming data corresponding to this first streaming data are received, then performing association processing of this first streaming data with its all correspondingly second streaming data.

[0075] Wherein, if the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index are not the same, which means only parts or none of the second streaming data corresponding to the first streaming data arrive at the server. Therefore, after all the second streaming data corresponding to the first streaming data arrive at the server, then performing association processing of both data.

Date Recue/Date Received 2021-10-18

[0076] To perform the association processing in time when the unarrived second streaming data corresponding to the first streaming data arrive at the server, if the server determines that the second streaming data corresponding to the first streaming data are not all received, wherein the first streaming data has correspondingly second streaming data, then performing following steps:

[0077] Determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;

[0078] Generating a second data index according to the first data index;

[0079] Storing the second data index and the complement data into the data storage middleware.

[0080] Wherein, the first data index and the second data index are not the same; the complement data is the second data identification excluded from the second data identification corresponding to the first data index in the second data identification corresponding to this first data identification. For example, the second data identifications corresponding to the first data identification are id 1, id 2, the second data identification corresponding to the first data index is id 1, then determining the complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index is id 2.

[0081] Specifically, when storing the second data index and the complement data to the data storage middleware, adopting a form of key-value to store the second data index and the complement data to the data storage middleware, wherein, the second data index is key, the complement data is value. If the complement data includes more than two second data identifications, using comma to separate each second data identification.

Date Recue/Date Received 2021-10-18

[0082] S130, after receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database.

[0083] In the present implementation, after the server receives the second streaming data, then judging whether its correspondingly first streaming data (hereinafter referred as the target first streaming data) has been received and whether all the second streaming data corresponding to the target first streaming data has been received.

[0084] In an implementation, judging whether the target first streaming data corresponding to the second streaming data has been received, the process is as follows:

[0085] Determining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;

[0086] Generating a second data index according to the first data identification corresponding to the second data identification;

[0087] Querying the data storage middleware according to the second data index;

[0088] If the second data identification corresponding to the second data index exists in the data storage middleware, determining of the target first streaming data corresponding to any of the second streaming data is received;

[0089] If the second data identification corresponding to the second data index does not exist in the data storage middleware, determining of the target first streaming data corresponding to Date Recue/Date Received 2021-10-18 any second streaming data is not received.

[0090] In the present implementation, a second streaming data includes a second data identification and the first data identification corresponding to this second identification (the target first data identification), therefore, after the server receives the second streaming data, obtaining the second data identification and the target first data identification from the second streaming data. Possibly, the second streaming data does not include the target first data identification, but the server or the data storage middleware maintain each first data identification in advance and the mapping relations between each first data identification and its correspondingly second identification, therefore, after the server obtains the second identification from the second streaming data, then obtaining the target first data identification corresponding to the second data identification through the mapping relations.
In the other implementations, a second streaming data can conclude plural second data identifications.

[0091] After obtaining the second data identification and its target first data identification, the server generates the second data index according to the target first data identification, then querying the data storage middleware through the generated second data index.

[0092] According to the above-mentioned, if the target first streaming data arrives at server earlier than the second streaming data, then the server will generate the second data index based on the first data identification included in the first streaming data, and storing this second data index and all the second data identification included in the first streaming data (such as storing in a key-value form) to the data storage middleware, therefore, if the first streaming data corresponding to the second streaming data has arrived at the server in advance, then the second data index generated from the present implementation can query relative results from the data storage middleware; Otherwise, if the first streaming data corresponding to the second streaming data has not arrived at the server, then querying data storage middleware through the second data index will not get any results, because this data storage middleware does not use the second data index (such as key) to store the data's records at this time.
Therefore, judging whether the target first streaming data is received or not through querying the data storage Date Recue/Date Received 2021-10-18 middleware by the second data index.

[0093] In an implementation, if the second data identification corresponding to the second data index does not exist in the data storage middleware, the server performs following steps:

[0094] Generating the first data index according to the target first data identification;

[0095] Storing the first data index and the second data identification corresponding to any second streaming data to the data storage middleware.

[0096] In the present implementation, according to the above-mentioned, if the data storage middleware does not exist the second data identification corresponding to the second data index, which means the first streaming data (the target first streaming data) corresponding to the second streaming data received by the server has not arrived, so the server can firstly determine the first data identification (the target first data identification) corresponding to the second data identification included in the second streaming data, then generating the first data index according to the target first data identification, then storing this first data index and the second data identification included in the second streaming data (such as storing in a key-value form) to the data storage middleware, and when the target first streaming data arrives, the server can perform relative processing.

[0097] In another hand, because the server can perform association processing after all the second streaming data corresponding to the target first streaming data arrive at the server, considering that the target first streaming data may correspond to plural second streaming data, therefore, the second streaming data received by the present server may not be the last one which arrives at the server in this plural second streaming data, or still existing parts of second streaming data corresponding to the target first streaming data will arrive at server after the current second streaming data arrives, therefore, if the present server queries data storage middleware by the current second streaming data to obtain the relative results (relative second data identification), then further judging if all the second streaming data corresponding to the Date Recue/Date Received 2021-10-18 target first streaming data have been received.

[0098] In an implementation, the process of judging whether all the second streaming data corresponding to the target first streaming data are all received, as following:

[0099] If the second data identification corresponding to the second data index exists in the data storage middleware, determining whether the second data identification and the second data identification corresponding to the second data index are the same;

[0100] If the second data identification and the second data identification corresponding to the second data index are the same, determining of all the second streaming data corresponding to the target first streaming data are received;

[0101] If the second data identification and the second data identification corresponding to the second data index are not the same, determining of all the second streaming data corresponding to the target first streaming data are not all received;

[0102] In the present implementation, according to the above-mentioned, in the data storage middleware, the second data index generated by the first data identification of the target first streaming data corresponds to all the second data identification included in the target first streaming data, to quickly determine if this second streaming data are the second streaming data corresponding to the target first streaming data or the last received second streaming data when receiving the second streaming data, in the present implementation, the server queries the relative second data identification from the data storage middleware according to the second data index, comparing all the queried second data identifications with the second data identification included in the received second streaming data, if the all the queried second data identifications and the second data identification included in the received second streaming data are not the same, then determining of all the second streaming data corresponding to the target first streaming data are not all received, at this time, then updating the results (the second data identification corresponding to the second data index) corresponding to the second data Date Recue/Date Received 2021-10-18 index in the data storage middleware, specifically, deleting the results corresponding to the second data index from the second data identification included in the received second streaming data in the data storage middleware. For example, the second data identifications corresponding to the second data index in the data storage middleware are id 1, id 2, but the second data identification included in the received second streaming data is id 1, and these data identifications are different after comparison, then updating the result corresponding to the second index from (id 1, id 2) to (id 2). However, if all the queried second data identifications and the second identification included in the received second streaming data are the same, determining of all the queried second data identification corresponding to the target first streaming data are all received, and then the server can perform association processing of the target first streaming data with all its correspondingly second streaming data.

[0103] In an implementation, in step S120, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, comprising:

[0104] Determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;

[0105] Determining a second amount corresponding to each second streaming data;

[0106] Obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, the third amount is the associated data after performing the association processing.

[0107] In an implementation, in step S130, performing association processing of the target first streaming data with all the corresponding second streaming data, comprising:

[0108] Determining the first amount corresponding to the target first streaming data;

Date Recue/Date Received 2021-10-18

[0109] Determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data;

[0110] Obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

[0111] In the above-mentioned two implementations, the server performs similar association processing on the target first streaming data (has correspondingly second streaming data) and its corresponding second streaming data.

[0112] For example, after the server receives a first streaming data which has correspondingly second streaming data, if determining of all the second streaming data corresponding to this first streaming data are all received, or after receiving a second streaming data, if determining of the first streaming data corresponding to this second streaming data is received and all the second streaming data corresponding to this first streaming data are all received, then the server determines the first amount corresponding to the first streaming data and the second amount of each second streaming data corresponding to this first streaming data, obtaining the third amount corresponding to the target first streaming data according to calculation of the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data.

[0113] Besides, after association processing, and/or when all the second streaming data corresponding to the first streaming data have arrived, deleting the first streaming data and all its correspondingly second streaming data in the data storage middleware to avoid the waste of storage space. Furthermore, after association processing, and/or when all the second streaming data corresponding to the first streaming data have arrived, deleting the first data index and second data index.
Date Recue/Date Received 2021-10-18

[0114] The following is a supportive illustration of the present invention's implementation through an order price control scenario.

[0115] In this scenario, the first data stream is order data stream, the first streaming data is order data that at least includes order number or order line number, if user uses coupon to purchase this order, then the order data also includes the coupon number and the coupon amount corresponding to his coupon number; the second data stream is promotion data stream, the second streaming data is promotion data that at least includes coupon number. The server access order data stream and promotion data stream through the stream processing platform Kafka, if any i orders of data and their correspondingly promotion data have arrived, the server performs association processing on both, but any orders of data and their correspondingly promotion data arrive at server randomly, therefore, there may be several kinds of situations:
1, the orders data arrive at server earlier than the promotion data; 2, the orders data arrive at server later than the promotion data; 3, when the orders data correspond to plural promotion data, the orders data arrive at server earlier than parts of the correspondingly promotion data.

[0116] Assuming that the order number (or order line number) included in orders data is order A, coupon numbers are coupon 1, coupon 2; the promotion data corresponding to this orders data is the first promotion data including the coupon number coupon 1 and the second promotion data including the coupon number coupon 2,

[0117] For the above-mentioned cases, the execution process can be as follows:

[0118] Assuming that the server receives the orders data first, the server will store the orders data to the data storage middleware Redis and obtain the orders data's first data identification (order number) orderA, because this orders data include coupon number coupon 1, coupon 2, the promotion data corresponding to this orders data can be determined, furthermore, the first data index prepare operate coupon orderA can be generated based on order A, then querying data storage middleware Redis through the first data index, because the orders data arrive at Date Recue/Date Received 2021-10-18 first, the query will not have any results at this time, so the server generates the second data index based on the first data identification, such as operating coupon orderA, and storing the second data index and coupon number in form of key-value to the Redis, wherein separating each coupon number by comma, such as (operating coupon orderA couponl, coupon2).

[0119] When the promotion data corresponding to the orders data arrived, the server obtains the second data identification from the received promotion data, if the first promotion data arrives, then storing the first promotion data to the Redis and obtaining its second data identification coupon 1 and its correspondingly first data identification orderA, the server will generate the second data index operating coupon orderA according to the first data identification orderA, and querying the second data identifications corresponding to the second data index by querying Redis which are couponl and coupon2, due to the difference between this promotion data's second data identification couponl and the second data identifications coupon 1 and coupon 2 corresponding to the second data index, which means this promotion data is not the last arriving promotion data of this orders data, therefore, updating the value corresponding to the second data index from couponl, coupon 2 to coupon 2 in Redis.
Furthermore, after second promotion data arrives, and after the above-mentioned process which can determine of all the promotion data corresponding to this orders data have all arrived at the server, then performing association processing of this orders data with their correspondingly promotion data which are the first promotion data and the second promotion data.

[0120] Except the coupon number included in each promotion data, activity information is also included, the server can judge (specific judge method is not limited in the present implementation) whether this coupon number is valid by this active information (can include activity status, collection method, activity times, etc.). Determining the order amount (or payment amount, which is the first amount) corresponding to the order number and the coupon amount (the second amount) corresponding to the coupon number by the pre-stored orders data, then summing up the order amount and the valid coupon amount of the coupon number to obtain the final amount (the third amount), comparing the final amount with the pre-set threshold, if the comparing result meets the pre-set condition, then determining of the order Date Recue/Date Received 2021-10-18 corresponding to the present order number is abnormal order, storing the order data corresponding to this order number to the database for the follow-up report displaying and/or alarm processing later (for example, sending alarm information to responsible personnel, etc.)

[0121] For the above-mentioned two cases, the server's execution process can be as follows:

[0122] Assuming that the server receives the first promotion data (can be the second promotion data) at first, the server will store the first promotion data to data storage middleware Redis and obtain the first promotion data's second data identification coupon 1 and its correspondingly first data identification orderA, then the second data index operating coupon orderA can be generated based on order A, then querying data storage middleware Redis through the second data index, because the orders data have not arrived yet, the query from Redis will not have any results at this time, so the server generates the first data index prepare operate coupon orderA based on the first data identification, and storing the first data index and coupon number couponl in form of key-value to the Redis, such as (prepare operate coupon orderA-coupon1).

[0123] Later when the second promotion data arrived, the server executes the above-mentioned process which can determine that the orders data is not arrived yet, this can incrementally update the value corresponding to the first data index, for example the result is (prepare operate coupon orderA couponl, coupon2).

[0124] At last when the orders data arrived, the server obtains the first data identification orderA from the orders data, and generates the first data index prepare operate coupon orderA based on orderA, and querying the data storage middleware Redis by the first data index, obtaining coupon number couponl and coupon2 at this time, the queried coupon number and the coupon number included in the orders data are the same, which means all the promotion data corresponding to this orders data have arrived, therefore, performing association processing, the association processing is as above-mentioned and will not be repeated here.

Date Recue/Date Received 2021-10-18

[0125] For the third case in the above-mentioned, the server's execution process can be as follows:

[0126] Assuming the server receives the first promotion data at first, the server will store the first promotion data to data storage middleware Redis and obtain the first promotion data's second data identification couponl and its correspondingly first data identification orderA, then the second data index operating coupon orderA can be generated based on order A, then querying data storage middleware Redis through the second data index, because the orders data have not arrived yet, the query from Redis will not have any results at this time, so the server generates the first data index prepare operate coupon orderA based on the first data identification, and storing the first data index and coupon number couponl in form of key-value to the Redis, such as (prepare operate coupon orderA-coupon1).

[0127] Later when server receives the orders data, storing the orders data to Redis and obtaining their first data identification orderA and coupon numbers couponl and coupon2, then generating the first data index prepare operate coupon orderA based on the first data identification orderA, obtaining the coupon number couponl by querying the Redis through the first data index, due to the difference between the second data identification included in the orders data and the second data identification corresponding to the first data index, which means the other promotion data have not arrived yet, therefore, the sever obtains the complement data between the second data identifications (coupon number couponl and coupon2) included in the orders data and the second data identification (coupon number coupon 1) corresponding to the first data index, which can obtain the coupon number coupon2, then generating the second data index operating coupon orderA based on the first data identification orderA, and storing the second data index and coupon number coupon2 in form of key-value to the Redis, such as (operating coupon orderA-coupon2).

[0128] When the second promotion data arrived, the server obtains the second data identifications from the second promotion data which are coupon number coupon2 and its Date Recue/Date Received 2021-10-18 correspondingly first data identification orderA, generating the second data index operating coupon orderA based on the first data identification, obtaining the second data identification corresponding to the second data index which is coupon number coupon2 by querying the data storage middleware Redis, at this time coupon number coupon2 included in the second promotion data and the coupon number of the second data identification corresponding to the second data are the same, which means all the promotion data corresponding to the orders data have arrived, therefore, performing association processing, the association processing is as above-mentioned and will not be repeated here.

[0129] In an implementation, as shown in Figure 4, a multi-data-stream processing apparatus is provided, including following modules:

[0130] A streaming data receiving module 110 configured to continuously receive the first streaming data and the second streaming data from a streaming processing platform, any first streaming data corresponds to zero or at least one second streaming data, any second streaming data corresponds to one first streaming data;

[0131] A first data processing module 120 configured to after receiving any receiving any first streaming data, wherein the first streaming data has correspondingly second streaming data, if determining of all the corresponding second streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;

[0132] A second data processing module 130 configured to after receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all Date Recue/Date Received 2021-10-18 the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database.

[0133] In an implementation, a first data processing module includes following sub modules:

[0134] A
first identification obtaining sub module configured to obtain a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;

[0135] A second identification determination sub module configured to determine a second data identification corresponding to the first data identification, the quantity of the second data identification corresponding to the first data identification is N, N is a positive integer greater than 0;

[0136] A first index generation sub module configured to generate a first data index according to the first data identification;

[0137] A query sub module configured to query the second data identification corresponding to the first data index in the data storage middleware, the quantity of the second data identification corresponding to the first data index is M, M is a positive integer greater than 0;

[0138] A determination sub module configured to when the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; when the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly Date Recue/Date Received 2021-10-18 second streaming data.

[0139] In an implementation, the first data processing module also includes following sub modules:

[0140] A complement data determination sub module configured to determine a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;

[0141] A second index generation sub module configured to generate a second data index according to the first data index;

[0142] A storage sub module configured to store the second data index and the complement data into the data storage middleware.

[0143] In an implementation, the second data processing module includes following sub modules:

[0144] A data identification determination sub module configured to determine the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;

[0145] A second index generation sub module configured to generate a second data index according to the first data identification corresponding to the second data identification;

[0146] A query sub module configured to query the data storage middleware according to the second data index;

[0147] A first determination sub module configured to when the second data identification Date Recue/Date Received 2021-10-18 corresponding to the second data index exists in the data storage middleware, determining of the target first streaming data corresponding to any of the second streaming data is received;
and when the second data identification corresponding to the second data index does not exist in the data storage middleware, determining of the target first streaming data corresponding to any second streaming data is not received.

[0148] In an implementation, the second data processing module also includes:

[0149] A
judging sub module configured to when the second data identification corresponding to the second data index exists in the data storage middleware, determining whether the second data identification is the same as the second data identification corresponding to the second data index;

[0150] A second determination sub module configured to when the second data identification and the second data identification corresponding to the second data index are the same, determining of all the second streaming data corresponding to the target first streaming data are received; and when the second data identification and the second data identification corresponding to the second data index are not the same, determining of all the second streaming data corresponding to the target first streaming data are not all received;

[0151] In an implementation, the second data processing module also includes:

[0152] A first index generation sub module configured to when the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification;

[0153] A storage sub module configured to store the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

[0154] The second data processing module also includes:

Date Recue/Date Received 2021-10-18

[0155] A data identification updating sub module configured to update the second data identification corresponding to the second data identification in the data storage middleware according to the second data identification.

[0156] In an implementation, the first data processing module also includes:

[0157] A first amount determination sub module configured to determine a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;

[0158] A second amount determination sub module configured to determine a second amount corresponding to each second streaming data;

[0159] A
association processing sub module configured to obtain a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, the third amount is the associated data after performing the association processing;

[0160] In an implementation, the second data processing module also includes:

[0161] A first amount determination sub module configured to determine the first amount corresponding to the target first streaming data;

[0162] A second amount determination sub module configured to determine the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data;

[0163] A association processing sub module configured to obtain the third amount Date Recue/Date Received 2021-10-18 corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

[0164] For the specific limitation of the multi-data-stream processing apparatus can refer to the above-mentioned the multi-data-stream processing method, which will not be repeated here.
Each module of the above data cache apparatus can be achieved fully or partly by software, hardware, and their combinations. The above modules can be embedded in the processor or independent of the processor in computer device and can store in the memory of computer device in form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0165] In an implementation, a computer device is provided to be a server and whose internal structure diagram is shown in Figure 4. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. The processor of the computer device is configured to provide calculation and control capabilities.
The memory of computer device includes non-volatile storage medium and internal memory. The memory of non-volatile storage medium has operation system, computer programs and database. The internal memory provides an environment for the operation system and computer program running in a non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection.
The computer program is executed by the processor to implement a multi-data-stream processing method.

[0166] The skilled in the art can understand that the structure shown in Figure 4 is only partial structural diagram related this application solution and not constitute limitation to the computer device applied on the current application solution, the specific computer device can include more or less components than what is shown in the figure, or combinations of some components or different components to what is shown in the figure.
Date Recue/Date Received 2021-10-18

[0167] In an implementation, a computer device is provided which includes a memory, a processor, and a computer program stored on the memory and running on the processor. The processor performs the following steps when executing the computer program:

[0168] Continuously receiving first streaming data and second streaming data from a streaming processing platform, any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data; after receiving any first streaming data, wherein the first streaming data has the correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data; and after receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database.

[0169] In an implementation, the processor executes the computer program, before achieving the step of performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, also achieving following steps:

[0170] Obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second data identification corresponding to the first data identification, the quantity of the second data identification corresponding to the first data identification is N, N is a positive integer greater Date Recue/Date Received 2021-10-18 than 0; generating a first data index according to the first data identification; querying the second data identification corresponding to the first data index in the data storage middleware, the quantity of the second data identification corresponding to the first data index is M, M is a positive integer greater than 0; if the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and if the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

[0171] In an implementation, the processor executes the computer program, after achieving the step of determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data, also achieving following steps:

[0172] Determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index; Generating a second data index according to the first data index; and storing the second data index and the complement data into the data storage middleware.

[0173] In an implementation, the processor executes the computer program, before achieving the step of performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, also achieving following steps:

[0174] Determining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;
generating a second data index according to the first data identification corresponding to the Date Recue/Date Received 2021-10-18 second data identification; querying the data storage middleware according to the second data index; if the second data identification corresponding to the second data index exists in the data storage middleware, determining of the target first streaming data corresponding to any of the second streaming data is received; and if the second data identification corresponding to the second data index does not exist in the data storage middleware, determining of the target first streaming data corresponding to any second streaming data is not received.

[0175] In an implementation, the processor executes the computer program to also achieve following steps:

[0176] If the second data identification corresponding to the second data index exists in the data storage middleware, determining whether the second data identification is the same as the second data identification corresponding to the second data index; if the second data identification and the second data identification corresponding to the second data index are the same, determining of all the second streaming data corresponding to the target first streaming data are received; and if the second data identification and the second data identification corresponding to the second data index are not the same, determining of all the second streaming data corresponding to the target first streaming data are not all received;

[0177] In an implementation, the processor executes the computer program to also achieve following steps:

[0178] If the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification; and storing the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

[0179] In an implementation, the processor executes the computer program, after achieving the step of determining of all the second streaming data corresponding to the target first streaming data are not all received, also achieving following steps:

Date Recue/Date Received 2021-10-18

[0180] According to the second data identification, updating the second data identification corresponding to the second data identification in the data storage middleware.

[0181] In an implementation, the processor executes the computer program to perform association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, also achieving following steps:

[0182] Determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data; determining a second amount corresponding to each second streaming data; and obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, the third amount is the associated data after performing the association processing.

[0183] In an implementation, the processor executes the computer program perform association processing of the target first streaming data with all the corresponding second streaming data, also achieving following steps:

[0184] Determining the first amount corresponding to the target first streaming data;
determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data; and obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing.

[0185] In an implementation, a computer readable storage medium stored with a computer Date Recue/Date Received 2021-10-18 program is provided to achieve the following steps when processor executes the computer program.

[0186] Continuously receiving first streaming data and second streaming data from a streaming processing platform, any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data; after receiving any first streaming data, wherein the first streaming data has the correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data; and after receiving any second streaming data, if the target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing, storing the target first streaming data into a database.

[0187] In an implementation, the processor executes the computer program, before achieving the step of performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, also achieving following steps:

[0188] Obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second data identification corresponding to the first data identification, the quantity of the second data identification corresponding to the first data identification is N, N is a positive integer greater than 0; generating a first data index according to the first data identification; querying the Date Recue/Date Received 2021-10-18 second data identification corresponding to the first data index in the data storage middleware, the quantity of the second data identification corresponding to the first data index is M, M is a positive integer greater than 0; if the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and if the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

[0189] In an implementation, the processor executes the computer program, after achieving the step of determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data, also achieving following steps:

[0190] Determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index; Generating a second data index according to the first data index; and storing the second data index and the complement data into the data storage middleware.

[0191] In an implementation, the processor executes the computer program, before achieving the step of performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, also achieving following steps:

[0192] Determining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;
generating a second data index according to the first data identification corresponding to the second data identification; querying the data storage middleware according to the second data Date Recue/Date Received 2021-10-18 index; if the second data identification corresponding to the second data index exists in the data storage middleware, determining of the target first streaming data corresponding to any of the second streaming data is received; and if the second data identification corresponding to the second data index does not exist in the data storage middleware, determining of the target first streaming data corresponding to any second streaming data is not received.

[0193] In an implementation, the processor executes the computer program to also achieve following steps:

[0194] If the second data identification corresponding to the second data index exists in the data storage middleware, determining whether the second data identification is the same as the second data identification corresponding to the second data index; if the second data identification and the second data identification corresponding to the second data index are the same, determining of all the second streaming data corresponding to the target first streaming data are received; and if the second data identification and the second data identification corresponding to the second data index are not the same, determining of all the second streaming data corresponding to the target first streaming data are not all received;

[0195] In an implementation, the processor executes the computer program to also achieve following steps:

[0196] If the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification; and storing the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

[0197] In an implementation, the processor executes the computer program, after achieving the step of determining of all the second streaming data corresponding to the target first streaming data are not all received, also achieving following steps:

Date Recue/Date Received 2021-10-18

[0198] According to the second data identification, updating the second data identification corresponding to the second data identification in the data storage middleware.

[0199] In an implementation, the processor executes the computer program to perform association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, also achieving following steps:

[0200] Determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data; determining a second amount corresponding to each second streaming data; and obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, the third amount is the associated data after performing the association processing.

[0201] In an implementation, the processor executes the computer program perform association processing of the target first streaming data with all the corresponding second streaming data, also achieving following steps:

[0202] Determining the first amount corresponding to the target first streaming data;
determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data; and obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing.

[0203] The skilled in the art can understand that all or partial of procedures from the above-mentioned methods can be performed by computer program instructions through related Date Recue/Date Received 2021-10-18 hardware, the mentioned computer program can be stored in a non-volatile material computer readable storage medium, this computer can include various implementation procedures from the abovementioned methods when execution. Any reference to the memory, the storage, the database, or the other media used in each implementation provided in current application can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programable RON (PROM), electrically programmable ROM (EPRPMD), electrically erasable programmable ROM(EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an instruction but not limited to, RAM is available in many forms such as static RAM (SRAM), dynamic RAM
(DRAMD), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SRAM (ESDRAM), synchronal link (Synchlink) DRAM (SLDRAM), memory bus (Rambus), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0204] The technical features of the above-mentioned implementations can be randomly combined, for concisely statement, not all possible combinations of technical features in the abovementioned implementations are described. However, if there are no conflicts in the combinations of these technical features, it shall be within the scope of this descriptions.

[0205] The above-mentioned implementations are only several implementations in this disclosure and the description is more specific and detailed but cannot be understood as the limitation of the scope of the invention patent. Evidently those ordinary skilled in the art can make various modifications and variations to the disclosure without departing from the spirit and scope of the disclosure. Therefore, the appended claims are intended to be construed as encompassing the described embodiment and all the modifications and variations coming into the scope of the disclosure.

Date Recue/Date Received 2021-10-18

Claims

Claims:

1. An apparatus comprising:
a streaming data receiving module configured to continuously receive a first streaming data and a second streaming data from a streaming processing platform, wherein any first streaming data corresponds to zero or at least one second streaming data, and any second streaming data corresponds to one first streaming data;
a first data processing module configured to:
wherein the first streaming data has correspondingly second streaming data, if determining of all the corresponding second streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when obtaining associated data meets preset conditions after the association processing;
storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;
a second data processing module configured to:
if the target first streaming data corresponding to any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, then performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing; and storing the target first streaming data into a database.

2. The apparatus of claim 1, wherein the first data processing module further comprises:
Date Recue/Date Received 2022-05-18 a first identification obtaining sub module configured to obtain a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;
a second identification determination sub module configured to determine a second data identification corresponding to the first data identification, wherein quantity of the second data identification corresponding to the first data identification is N, and wherein N is a positive integer greater than 0;
a first index generation sub module configured to generate a first data index according to the first data identification;
a query sub module configured to query the second data identification corresponding to the first data index in a data storage middleware, wherein quantity of the second data identification corresponding to the first data index is M, and wherein M is a positive integer greater than 0;
a determination sub module configured to:
when the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and when the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining of all the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

3. The apparatus of claim 2, wherein the first data processing module further comprises:

Date Recue/Date Received 2022-05-18 a complement data determination sub module configured to determine a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;
a second index generation sub module configured to generate a second data index according to the first data index; and a storage sub module configured to store the second data index and the complement data into the data storage middleware.

4. The apparatus of claim 1, wherein the second data processing module further comprises:
a data identification determination sub module configured to detennine the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;
a second index generation sub module configured to generate a second data index according to the first data identification corresponding to the second data identification;
a query sub module configured to query data storage middleware according to the second data index;
a first detennination sub module configured to:
when the second data identification corresponding to the second data index exists in the data storage middleware, determining of the target first streaming data corresponding to any of the second streaming data is received; and when the second data identification corresponding to the second data index does not exist in the data storage middleware, determining of the target first streaming data corresponding to any second streaming data is not received.

5. The apparatus of claim 4, wherein the second data processing module further comprises:

Date Recue/Date Received 2022-05-18 a judging sub module configured to when the second data identification corresponding to the second data index exists in the data storage middleware, determining the second data identification is the same as the second data identification corresponding to the second data index;
a second determination sub module configured to:
when the second data identification and the second data identification corresponding to the second data index are the same, determining of all the second streaming data corresponding to the target first streaming data are received; and when the second data identification and the second data identification corresponding to the second data index are not the same, determining of all the second streaming data corresponding to the target first streaming data are not all received;

6. The apparatus of claim 5, wherein the second data processing module further comprises:
a first index generation sub module configured to when the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification; and a storage sub module configured to store the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

7. The apparatus of claim 6, wherein the second data processing module further comprises:
a data identification updating sub module configured to update the second data identification corresponding to the second data identification in the data storage middleware according to the second data identification.

8. The apparatus of claim 1, wherein the first data processing module further comprises:

Date Recue/Date Received 2022-05-18 a first amount determination sub module configured to determine a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;
a second amount deteimination sub module configured to determine a second amount corresponding to each second streaming data; and a association processing sub module configured to obtain a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, the third amount is the associated data after performing the association processing;

9. The apparatus of claim 8, wherein the second data processing module further comprises:
a first amount determination sub module configured to determine the first amount corresponding to the target first streaming data;
a second amount deteimination sub module configured to determine the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data; and an association processing sub module configured to obtain the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

10. The apparatus of any one of claims 1 to 9, wherein data of a first data stream is the first streaming data, and data of a second data steam is the second streaming data.

11. The apparatus of any one of claims 1 to 10, wherein the first streaming data includes the first data identification, and the second streaming data includes the second data identification.

Date Recue/Date Received 2022-05-18

12. The apparatus of any one of claims 1 to 11, wherein there is correspondence relations between the first streaming data from the first data stream and one or plural second streaming data from the second data stream.

13. The apparatus of any one of claims 1 to 12, stores the first streaming data and the second streaming data sent by a server to the data storage middleware.

14. The apparatus of any one of claims 1 to 13, is implemented as independent server or as a cluster server composed of plural servers.

15. The apparatus of any one of claims 1 to 14, the data storage middleware and the database are deployed on the same server or a cluster of servers

16. The apparatus of any one of claims 1 to 15, the data storage middleware and the database are deployed on an independent server or a cluster of servers.

17. The apparatus of any one of claims 1 to 16, wherein the first streaming data which has correspondingly second data identification, the first streaming data not only has the first data identification but has all second data identifications.

18. The apparatus of any one of claims 1 to 17, wherein any first streaming data from the first data stream, if existing the second streaming data corresponding to the first streaming data in the second data stream, for the any first streaming data and the second streaming data corresponding to the first streaming data, their order of arriving is random.

19. The apparatus of any one of claims 1 to 18, wherein the first streaming data earlier than the second streaming data and arrives later then all the other second streaming data.

20. The apparatus of any one of claims 1 to 19, wherein if the first streaming data has correspondingly two or more second streaming data, the first streaming data arrives later than parts of its correspondingly second streaming data, but arriving earlier that the rest parts of its correspondingly second streaming data.
Date Recue/Date Received 2022-05-18

21. The apparatus of any one of claims 1 to 20, wherein performs association processing only after any first streaming data and all its correspondingly second streaming data arrives.

22. The apparatus of any one of claims 1 to 21, wherein a judging apparatus is judging existing the second data identification except the first data identification in the first streaming data.

23. The apparatus of any one of claims 1 to 22, wherein if the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index are not the same, wherein only parts or none of the second streaming data corresponding to the first streaming data arrived.

24. The apparatus of any one of claims 1 to 23, wherein all the second streaming data corresponding to the first streaming data arrives, performing association processing of both data.

25. The apparatus of any one of claims 1 to 24, wherein the first data index and the second data index are not the same;

26. The apparatus of any one of claims 1 to 25, wherein the complement data is the second data identification excluded from the second data identification corresponding to the first data index in the second data identification corresponding to this first data identification.

27. The apparatus of any one of claims 1 to 26, wherein when storing the second data index and the complement data to the data storage middleware, adopting a form of key-value to store the second data index and the complement data to the data storage middleware, wherein the second data index is key, the complement data is value.

28. The apparatus of any one of claims 1 to 27, wherein if the complement data includes more than two second data identifications, using comma to separate each second data identification.

Date Recue/Date Received 2022-05-18

29. The apparatus of any one of claims 1 to 28, wherein the second streaming data does not include the target first data identification, the database or the data storage middleware maintain each first data identification in advance and mapping relations between each first data identification and its correspondingly second identification.

30. The apparatus of any one of claims 1 to 29, wherein the second streaming data concludes plural of second data identifications.

31. The apparatus of any one of claims 1 to 30, wherein if the target first streaming data arrives earlier than the second streaming data, generate the second data index based on the first data identification included in the first streaming data, and storing the second data index and all the second data identification included in the first streaming data, in a key-value form, to the data storage middleware.

32. The apparatus of any one of claims 1 to 31, wherein if the first streaming data corresponding to the second streaming data has arrived in advance, then the second data index generated queries relative results from the data storage middleware;

33. The apparatus of any one of claims 1 to 32, wherein if the first streaming data corresponding to the second streaming data has not arrived, then querying data storage middleware through the second data index does not get any results, because this data storage middleware does not use the second data index to store the data's records at this time.

34. The apparatus of any one of claims 1 to 33, wherein judging the target first streaming data is received or not through querying the data storage middleware by the second data index.

35. The apparatus of any one of claims 1 to 34, further includes deleting the results corresponding to the second data index from the second data identification included in the received second streaming data in the data storage middleware.

36. The apparatus of any one of claims 1 to 35, further includes deleting the first streaming data and all its correspondingly second streaming data in the data storage middleware.

Date Recue/Date Received 2022-05-18

37. The apparatus of any one of claims 1 to 36, further includes deleting the first data index and second data index.

38. A method comprising:
continuously receiving first streaming data and second streaming data from a streaming processing platform, wherein any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data;
wherein the first streaming data has correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, storing any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;
if target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing; and storing the target first streaming data into a database.

39. The method of claim 38, further comprising:
obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;

Date Recue/Date Received 2022-05-18 determining a second data identification corresponding to the first data identification, wherein the quantity of the second data identification corresponding to the first data identification is N, and wherein N is a positive integer greater than 0;
generating a first data index according to the first data identification;
querying the second data identification corresponding to the first data index in data storage middleware, wherein quantity of the second data identification corresponding to the first data index is M, and wherein M is a positive integer greater than 0;
if the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining all of the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and if the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining all of the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

40. The method of claim 39, further comprising:
determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;
generating a second data index according to the first data index; and storing the second data index and the complement data into the data storage middleware.

41. The method of claim 38, further comprising:
determining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;

Date Recue/Date Received 2022-05-18 generating a second data index according to the first data identification corresponding to the second data identification;
querying data storage middleware according to the second data index;
if the second data identification corresponding to the second data index exists in the data storage middleware, determining the target first streaming data corresponding to any of the second streaming data is received; and if the second data identification corresponding to the second data index does not exist in the data storage middleware, determining the target first streaming data corresponding to any second streaming data is not received.

42. The method of claim 41, further comprising:
if the second data identification corresponding to the second data index exists in the data storage middleware, determining the second data identification is the same as the second data identification corresponding to the second data index;
if the second data identification and the second data identification corresponding to the second data index are the same, determining all the second streaming data corresponding to the target first streaming data are received; and if the second data identification and the second data identification corresponding to the second data index are not the same, determining all the second streaming data corresponding to the target first streaming data are not all received.

43. The method of claim 42, further comprising:
if the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification; and storing the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.
Date Recue/Date Received 2022-05-18

44. The method of claim 43, further comprising:
according to the second data identification, updating the second data identification corresponding to the second data identification in the data storage middleware.

45. The method of claim 38, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, comprising:
determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second amount corresponding to each second streaming data; and obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, and the third amount is the associated data after performing the association processing;

46. The method of claim 45, wherein performing association processing of the target first streaming data with all the corresponding second streaming data, comprises:
determining the first amount corresponding to the target first streaming data;
determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data; and obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

47. The method of any one of claims 38 to 46, wherein data of a first data stream is the first streaming data, and data of a second data steam is the second streaming data.

Date Recue/Date Received 2022-05-18

48. The method of any one of claims 38 to 47, wherein the first streaming data includes the first data identification, and the second streaming data includes the second data identification.

49. The method of any one of claims 38 to 48, wherein there is correspondence relations between the first streaming data from the first data stream and one or plural second streaming data from the second data stream.

50. The method of any one of claims 38 to 49, stores the first streaming data and the second streaming data sent by a server to the data storage middleware.

51. The method of any one of claims 38 to 50, is implemented as independent server or as a cluster server composed of plural servers.

52. The method of any one of claims 38 to 51, the data storage middleware and the database are deployed on the same server or a cluster of servers

53. The method of any one of claims 38 to 52, the data storage middleware and the database are deployed on an independent server or a cluster of servers.

54. The method of any one of claims 38 to 53, wherein the first streaming data which has correspondingly second data identification, the first streaming data not only has the first data identification but has all second data identifications.

55. The method of any one of claims 38 to 54, wherein any first streaming data from the first data stream, if existing the second streaming data corresponding to the first streaming data in the second data stream, for the any first streaming data and the second streaming data corresponding to the first streaming data, their order of arriving is random.

56. The method of any one of claims 38 to 55, wherein the first streaming data earlier than the second streaming data and arrives later then all the other second streaming data.

Date Recue/Date Received 2022-05-18

57. The method of any one of claims 38 to 56, wherein if the first streaming data has correspondingly two or more second streaming data, the first streaming data arrives later than parts of its correspondingly second streaming data, but arriving earlier that the rest parts of its correspondingly second streaming data.

58. The method of any one of claims 38 to 57, wherein performs association processing only after any first streaming data and all its correspondingly second streaming data arrives.

59. The method of any one of claims 38 to 58, wherein a judging method is judging existing the second data identification except the first data identification in the first streaming data.

60. The method of any one of claims 38 to 59, wherein if the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index are not the same, wherein only parts or none of the second streaming data corresponding to the first streaming data arrived.

61. The method of any one of claims 38 to 60, wherein all the second streaming data corresponding to the first streaming data arrives, performing association processing of both data.

62. The method of any one of claims 38 to 61, wherein the first data index and the second data index are not the same;

63. The method of any one of claims 38 to 62, wherein the complement data is the second data identification excluded from the second data identification corresponding to the first data index in the second data identification corresponding to this first data identification.

64. The method of any one of claims 38 to 63, wherein when storing the second data index and the complement data to the data storage middleware, adopting a form of key-value to store the second data index and the complement data to the data storage middleware, wherein the second data index is key, the complement data is value.

Date Recue/Date Received 2022-05-18

65. The method of any one of claims 38 to 64, wherein if the complement data includes more than two second data identifications, using comma to separate each second data identification.

66. The method of any one of claims 38 to 65, wherein the second streaming data does not include the target first data identification, the database or the data storage middleware maintain each first data identification in advance and mapping relations between each first data identification and its correspondingly second identification.

67. The method of any one of claims 38 to 66, wherein the second streaming data concludes plural of second data identifications.

68. The method of any one of claims 38 to 67, wherein if the target first streaming data arrives earlier than the second streaming data, generate the second data index based on the first data identification included in the first streaming data, and storing the second data index and all the second data identification included in the first streaming data, in a key-value fonn, to the data storage middleware.

69. The method of any one of claims 38 to 68, wherein if the first streaming data corresponding to the second streaming data has arrived in advance, then the second data index generated queries relative results from the data storage middleware;

70. The method of any one of claims 38 to 69, wherein if the first streaming data corresponding to the second streaming data has not arrived, then querying data storage middleware through the second data index does not get any results, because this data storage middleware does not use the second data index to store the data's records at this time.

71. The method of any one of claims 38 to 70, wherein judging the target first streaming data is received or not through querying the data storage middleware by the second data index.

72. The method of any one of claims 38 to 71, further includes deleting the results corresponding to the second data index from the second data identification included in the received second streaming data in the data storage middleware.

Date Recue/Date Received 2022-05-18

73. The method of any one of claims 38 to 72, further includes deleting the first streaming data and all its correspondingly second streaming data in the data storage middleware.

74. The method of any one of claims 38 to 73, further includes deleting the first data index and second data index.

75. A device comprising:
a memory, includes non-volatile storage medium and internal memory, wherein the memory of non-volatile storage medium has operation system, computer programs and database, wherein the internal memory provides an environment for the operation system and the computer program running in the non-volatile storage medium;
a network interface, configured to communicate with an external terminal through a network connection;
the database connected through a system bus;
a processor, configured to provide calculation and control capabilities;
the computer program stored in the memory and executed on the processor configured to:
continuously receive first streaming data and second streaming data from a streaming processing platform, wherein any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data;
Date Recue/Date Received 2022-05-18 wherein the first streaming data has correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, store any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;
if target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing; and store the target first streaming data into a database.

76. The device of claim 75, further comprising:
obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second data identification corresponding to the first data identification, wherein the quantity of the second data identification corresponding to the first data identification is N, and wherein N is a positive integer greater than 0;
generating a first data index according to the first data identification;
querying the second data identification corresponding to the first data index in data storage middleware, wherein quantity of the second data identification corresponding to the first data index is M, and wherein M is a positive integer greater than 0;

Date Recue/Date Received 2022-05-18 if the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining all of the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and if the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining all of the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

77. The device of claim 76, further comprising:
determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;
generating a second data index according to the first data index; and storing the second data index and the complement data into the data storage middleware.

78. The device of claim 75, further comprising:
determining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;
generating a second data index according to the first data identification corresponding to the second data identification;
querying data storage middleware according to the second data index;
if the second data identification corresponding to the second data index exists in the data storage middleware, determining the target first streaming data corresponding to any of the second streaming data is received; and Date Recue/Date Received 2022-05-18 if the second data identification corresponding to the second data index does not exist in the data storage middleware, determining the target first streaming data corresponding to any second streaming data is not received.

79. The device of claim 78, further comprising:
if the second data identification corresponding to the second data index exists in the data storage middleware, determining the second data identification is the same as the second data identification corresponding to the second data index;
if the second data identification and the second data identification corresponding to the second data index are the same, determining all the second streaming data corresponding to the target first streaming data are received; and if the second data identification and the second data identification corresponding to the second data index are not the same, determining all the second streaming data corresponding to the target first streaming data are not all received.

80. The device of claim 79, further comprising:
if the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification; and storing the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

81. The device of claim 80, further comprising:
according to the second data identification, updating the second data identification corresponding to the second data identification in the data storage middleware.

82. The device of claim 75, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, comprising:

Date Recue/Date Received 2022-05-18 determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second amount corresponding to each second streaming data; and obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, and the third amount is the associated data after performing the association processing;

83. The device of claim 82, wherein performing association processing of the target first streaming data with all the corresponding second streaming data, comprises:
determining the first amount corresponding to the target first streaming data;
determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data; and obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

84. The device of any one of claims 75 to 83, wherein data of a first data stream is the first streaming data, and data of a second data steam is the second streaming data.

85. The device of any one of claims 75 to 84, wherein the first streaming data includes the first data identification, and the second streaming data includes the second data identification.

86. The device of any one of claims 75 to 85, wherein there is correspondence relations between the first streaming data from the first data stream and one or plural second streaming data from the second data stream.

87. The device of any one of claims 75 to 86, stores the first streaming data and the second streaming data sent by a server to the data storage middleware.

Date Recue/Date Received 2022-05-18

88. The device of any one of claims 75 to 87, is implemented as independent server or as a cluster server composed of plural servers.

89. The device of any one of claims 75 to 88, the data storage middleware and the database are deployed on the same server or a cluster of servers

90. The device of any one of claims 75 to 89, the data storage middleware and the database are deployed on an independent server or a cluster of servers.

91. The device of any one of claims 75 to 90, wherein the first streaming data which has correspondingly second data identification, the first streaming data not only has the first data identification but has all second data identifications.

92. The device of any one of claims 75 to 91, wherein any first streaming data from the first data stream, if existing the second streaming data corresponding to the first streaming data in the second data stream, for the any first streaming data and the second streaming data corresponding to the first streaming data, their order of arriving is random.

93. The device of any one of claims 75 to 92, wherein the first streaming data earlier than the second streaming data and arrives later then all the other second streaming data.

94. The device of any one of claims 75 to 93, wherein if the first streaming data has correspondingly two or more second streaming data, the first streaming data arrives later than parts of its correspondingly second streaming data, but arriving earlier that the rest parts of its correspondingly second streaming data.

95. The device of any one of claims 75 to 94, wherein performs association processing only after any first streaming data and all its correspondingly second streaming data arrives.

96. The device of any one of claims 75 to 95, wherein a judging device is judging existing the second data identification except the first data identification in the first streaming data.
Date Recue/Date Received 2022-05-18

97. The device of any one of claims 75 to 96, wherein if the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index are not the same, wherein only parts or none of the second streaming data corresponding to the first streaming data arrived.

98. The device of any one of claims 75 to 97, wherein all the second streaming data corresponding to the first streaming data arrives, performing association processing of both data.

99. The device of any one of claims 75 to 98, wherein the first data index and the second data index are not the same;

100. The device of any one of claims 75 to 99, wherein the complement data is the second data identification excluded from the second data identification corresponding to the first data index in the second data identification corresponding to this first data identification.

101. The device of any one of claims 75 to 100, wherein when storing the second data index and the complement data to the data storage middleware, adopting a form of key-value to store the second data index and the complement data to the data storage middleware, wherein the second data index is key, the complement data is value.

102. The device of any one of claims 75 to 101, wherein if the complement data includes more than two second data identifications, using comma to separate each second data identification.

103. The device of any one of claims 75 to 102, wherein the second streaming data does not include the target first data identification, the database or the data storage middleware maintain each first data identification in advance and mapping relations between each first data identification and its correspondingly second identification.

104. The device of any one of claims 75 to 103, wherein the second streaming data concludes plural of second data identifications.

Date Recue/Date Received 2022-05-18

105. The device of any one of claims 75 to 104, wherein if the target first streaming data arrives earlier than the second streaming data, generate the second data index based on the first data identification included in the first streaming data, and storing the second data index and all the second data identification included in the first streaming data, in a key-value form, to the data storage middleware.

106. The device of any one of claims 75 to 105, wherein if the first streaming data corresponding to the second streaming data has arrived in advance, then the second data index generated queries relative results from the data storage middleware;

107. The device of any one of claims 75 to 106, wherein if the first streaming data corresponding to the second streaming data has not arrived, then querying data storage middleware through the second data index does not get any results, because this data storage middleware does not use the second data index to store the data's records at this time.

108. The device of any one of claims 75 to 107, wherein judging the target first streaming data is received or not through querying the data storage middleware by the second data index.

109. The device of any one of claims 75 to 108, further includes deleting the results corresponding to the second data index from the second data identification included in the received second streaming data in the data storage middleware.

110. The device of any one of claims 75 to 109, further includes deleting the first streaming data and all its correspondingly second streaming data in the data storage middleware.

111. The device of any one of claims 75 to 110, further includes deleting the first data index and second data index.

112. A computer readable physical memory having stored thereon a computer program executed by a computer configured to:

Date Recue/Date Received 2022-05-18 continuously receive first streaming data and second streaming data from a streaming processing platform, wherein any first streaming data corresponds to zero second streaming data or at least one second streaming data, and any second streaming data corresponds to one first streaming data;
wherein the first streaming data has correspondingly second streaming data, if all the second streaming data corresponding to the first streaming data are received, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, when the obtaining associated data meets preset conditions after the association processing, store any first streaming data into a database, wherein the first streaming data has the correspondingly second streaming data;
if target first streaming data corresponding to the any second streaming data is received, and all the second streaming data corresponding to the target first streaming data are received, performing association processing of the target first streaming data with all the second streaming data corresponding to the target first streaming data, when the obtaining associated data meets preset conditions after the association processing; and store the target first streaming data into a database.

113. The memory of claim 112, further comprising:
obtaining a first data identification included in any first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second data identification corresponding to the first data identification, wherein the quantity of the second data identification corresponding to the first data identification is N, and wherein N is a positive integer greater than 0;
generating a first data index according to the first data identification;

Date Recue/Date Received 2022-05-18 querying the second data identification corresponding to the first data index in data storage middleware, wherein quantity of the second data identification corresponding to the first data index is M, and wherein M is a positive integer greater than 0;
if the second data identification corresponding to the first data identification is the same as the second data identification corresponding to the first data index, determining all of the second streaming data corresponding to any first streaming data are received, wherein the first streaming data has the correspondingly second streaming data; and if the second data identification corresponding to the first data identification is not the same as the second data identification corresponding to the first data index, determining all of the second streaming data corresponding to any first streaming data are not all received, wherein the first streaming data has the correspondingly second streaming data.

114. The memory of claim 113, further comprising:
determining a complement data between the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index;
generating a second data index according to the first data index; and storing the second data index and the complement data into the data storage middleware.

115. The memory of claim 112, further comprising:
detennining the second data identification corresponding to any second streaming data, and a target first data identification corresponding to the second data identification;
generating a second data index according to the first data identification corresponding to the second data identification;
querying data storage middleware according to the second data index;

Date Recue/Date Received 2022-05-18 if the second data identification corresponding to the second data index exists in the data storage middleware, determining the target first streaming data corresponding to any of the second streaming data is received; and if the second data identification corresponding to the second data index does not exist in the data storage middleware, determining the target first streaming data corresponding to any second streaming data is not received.

116. The memory of claim 115, further comprising:
if the second data identification corresponding to the second data index exists in the data storage middleware, determining the second data identification is the same as the second data identification corresponding to the second data index;
if the second data identification and the second data identification corresponding to the second data index are the same, determining all the second streaming data corresponding to the target first streaming data are received; and if the second data identification and the second data identification corresponding to the second data index are not the same, determining all the second streaming data corresponding to the target first streaming data are not all received.

117. The memory of claim 116, further comprising:
if the second data identification corresponding to the second data index does not exist in the data storage middleware, generating a first data index according to the target first data identification; and storing the first data index and the second data identification corresponding to any second streaming data into the data storage middleware.

118. The memory of claim 117, further comprising:
according to the second data identification, updating the second data identification corresponding to the second data identification in the data storage middleware.
Date Recue/Date Received 2022-05-18

119. The memory of claim 112, performing association processing of any first streaming data with all the second streaming data corresponding to the first streaming data, wherein the first streaming data has the correspondingly second streaming data, comprising:
determining a first amount of the first streaming data, wherein the first streaming data has the correspondingly second streaming data;
determining a second amount corresponding to each second streaming data; and obtaining a third amount corresponding to any first streaming data according to the first amount and the second amount corresponding to each second streaming data, wherein the first streaming data has the correspondingly second streaming data, and the third amount is the associated data after performing the association processing;

120. The memory of claim 119, wherein performing association processing of the target first streaming data with all the corresponding second streaming data, comprises:
determining the first amount corresponding to the target first streaming data;
determining the second amount corresponding to each second streaming data, wherein the second streaming data corresponding to the target first streaming data; and obtaining the third amount corresponding to the target first streaming data according to the first amount corresponding to the target first streaming data and the second amount corresponding to each second streaming data, wherein the third amount is the associated data after performing the association processing;

121. The memory of any one of claims 112 to 120, wherein data of a first data stream is the first streaming data, and data of a second data steam is the second streaming data.

122. The memory of any one of claims 112 to 121, wherein the first streaming data includes the first data identification, and the second streaming data includes the second data identification.

Date Recue/Date Received 2022-05-18

123. The memory of any one of claims 112 to 122, wherein there is correspondence relations between the first streaming data from the first data stream and one or plural second streaming data from the second data stream.

124. The memory of any one of claims 112 to 123, stores the first streaming data and the second streaming data sent by a server to the data storage middleware.

125. The memory of any one of claims 112 to 124, is implemented as independent server or as a cluster server composed of plural servers.

126. The memory of any one of claims 112 to 125, the data storage middleware and the database are deployed on the same server or a cluster of servers

127. The memory of any one of claims 112 to 126, the data storage middleware and the database are deployed on an independent server or a cluster of servers.

128. The memory of any one of claims 112 to 127, wherein the first streaming data which has correspondingly second data identification, the first streaming data not only has the first data identification but has all second data identifications.

129. The memory of any one of claims 112 to 128, wherein any first streaming data from the first data stream, if existing the second streaming data corresponding to the first streaming data in the second data stream, for the any first streaming data and the second streaming data corresponding to the first streaming data, their order of arriving is random.

130. The memory of any one of claims 112 to 129, wherein the first streaming data earlier than the second streaming data and arrives later then all the other second streaming data.

131. The memory of any one of claims 112 to 130, wherein if the first streaming data has correspondingly two or more second streaming data, the first streaming data arrives later than parts of its correspondingly second streaming data, but arriving earlier that the rest parts of its correspondingly second streaming data.

Date Recue/Date Received 2022-05-18

132. The memory of any one of claims 112 to 131, wherein performs association processing only after any first streaming data and all its correspondingly second streaming data arrives.

133. The memory of any one of claims 112 to 132, wherein a judging memory is judging existing the second data identification except the first data identification in the first streaming data.

134. The memory of any one of claims 112 to 133, wherein if the second data identification corresponding to the first data identification and the second data identification corresponding to the first data index are not the same, wherein only parts or none of the second streaming data corresponding to the first streaming data arrived.

135. The memory of any one of claims 112 to 134, wherein all the second streaming data corresponding to the first streaming data arrives, performing association processing of both data.

136. The memory of any one of claims 112 to 135, wherein the first data index and the second data index are not the same;

137. The memory of any one of claims 112 to 136, wherein the complement data is the second data identification excluded from the second data identification corresponding to the first data index in the second data identification corresponding to this first data identification.

138. The memory of any one of claims 112 to 137, wherein when storing the second data index and the complement data to the data storage middleware, adopting a form of key-value to store the second data index and the complement data to the data storage middleware, wherein the second data index is key, the complement data is value.

139. The memory of any one of claims 112 to 138, wherein if the complement data includes more than two second data identifications, using comma to separate each second data identification.

Date Recue/Date Received 2022-05-18

140. The memory of any one of claims 112 to 139, wherein the second streaming data does not include the target first data identification, the database or the data storage middleware maintain each first data identification in advance and mapping relations between each first data identification and its correspondingly second identification.

141. The memory of any one of claims 112 to 140, wherein the second streaming data concludes plural of second data identifications.

142. The memory of any one of claims 112 to 141, wherein if the target first streaming data arrives earlier than the second streaming data, generate the second data index based on the first data identification included in the first streaming data, and storing the second data index and all the second data identification included in the first streaming data, in a key-value form, to the data storage middleware.

143. The memory of any one of claims 112 to 142, wherein if the first streaming data corresponding to the second streaming data has arrived in advance, then the second data index generated queries relative results from the data storage middleware;

144. The memory of any one of claims 112 to 143, wherein if the first streaming data corresponding to the second streaming data has not arrived, then querying data storage middleware through the second data index does not get any results, because this data storage middleware does not use the second data index to store the data's records at this time.

145. The memory of any one of claims 112 to 144, wherein judging the target first streaming data is received or not through querying the data storage middleware by the second data index.

146. The memory of any one of claims 112 to 145, further includes deleting the results corresponding to the second data index from the second data identification included in the received second streaming data in the data storage middleware.

147. The memory of any one of claims 112 to 146, further includes deleting the first streaming data and all its correspondingly second streaming data in the data storage middleware.

Date Recue/Date Received 2022-05-18

148. The memory of any one of claims 112 to 147, further includes deleting the first data index and second data index.
Date Recue/Date Received 2022-05-18