WO2021169674A1

WO2021169674A1 - Processing method and apparatus for service data

Info

Publication number: WO2021169674A1
Application number: PCT/CN2021/072677
Authority: WO
Inventors: 吴武桓; 孙瑞明; 张磊
Original assignee: 京东数字科技控股股份有限公司
Priority date: 2020-02-28
Filing date: 2021-01-19
Publication date: 2021-09-02
Also published as: CN111367688A

Abstract

The present invention relates to a processing method and apparatus for service data. The method comprises: querying, from a service module, the data information of service data to be processed by the service module; according to the data information, generating task messages having one-to-one correspondence to the service data; and transmitting the task messages to the service module by means of a message queue, wherein the task messages are used for indicating the service module to process the service data corresponding to the task messages. The present invention solves the technical problem in the related technology of the low processing efficiency of the service data.

Description

一种业务数据的处理方法和装置Method and device for processing business data

本公开要求于2020年02月28日提交中国专利局、优先权号为202010131312.1、发明名称为“一种业务数据的处理方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。This disclosure claims the priority of a Chinese patent application filed with the Chinese Patent Office on February 28, 2020, the priority number is 202010131312.1, and the invention title is "a method and device for processing business data", the entire content of which is incorporated by reference In this disclosure.

技术领域Technical field

本公开涉及计算机领域，尤其涉及一种业务数据的处理方法和装置。The present disclosure relates to the field of computers, and in particular to a method and device for processing business data.

背景技术Background technique

大部分任务***处理海量数据是处理自有的任务数据，但某些场景需要任务***定时触发分布式业务***，由业务***自身处理海量业务数据，业务***需保证并发处理能力与数据最终正确性。例如，某些任务***需要定时在月底去触发业务***处理海量的业务数据，业务***需要去保证业务处理的并发能力，还有数据最终的正确性。Most task systems process massive data by processing their own task data, but in some scenarios, the task system needs to trigger the distributed business system regularly, and the business system itself processes the massive business data. The business system needs to ensure the concurrent processing capability and the final correctness of the data. . For example, certain task systems need to trigger the business system to process massive amounts of business data regularly at the end of the month, and the business system needs to ensure the concurrency of business processing and the final correctness of the data.

任务***定时通知分布式业务***，业务***处理对应的数据直到数据全部处理完成。在实现本发明过程中，发明人发现这种数据一般规模都比较庞大，会导致业务***并发压力过大，可能会影响正常业务服务。因此可能会导致业务数据的处理效率的下降。The task system regularly informs the distributed business system, and the business system processes the corresponding data until all the data is processed. In the process of implementing the present invention, the inventor found that this kind of data is generally large in scale, which will cause excessive pressure on the concurrency of the business system, and may affect normal business services. Therefore, the processing efficiency of business data may decrease.

针对上述的问题，目前尚未提出有效的解决方案。In view of the above-mentioned problems, no effective solutions have yet been proposed.

发明内容Summary of the invention

本公开提供了一种业务数据的处理方法和装置，以至少解决相关技术中业务数据的处理效率较低的技术问题。The present disclosure provides a method and device for processing business data to at least solve the technical problem of low processing efficiency of business data in related technologies.

根据本公开实施例的一个方面，提供了一种业务数据的处理方法，包括：According to an aspect of the embodiments of the present disclosure, there is provided a method for processing business data, including:

从业务模块中查询待所述业务模块处理的业务数据的数据信息；Query data information of the business data to be processed by the business module from the business module;

根据所述数据信息生成与所述业务数据一一对应的任务消息；Generating task messages corresponding to the business data one-to-one according to the data information;

将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。The task message is sent to the service module through a message queue, where the task message is used to instruct the service module to process the service data corresponding to the task message.

可选地，所述数据信息包括所述业务数据的数据总量，其中，根据所述数据信息生成与所述业务数据一一对应的所述任务消息包括：Optionally, the data information includes the total amount of data of the service data, and generating the task message corresponding to the service data one-to-one according to the data information includes:

将所述数据总量划分为多个分页数据量；Dividing the total amount of data into multiple paging data amounts;

创建所述多个分页数据量一一对应的多个分页任务；Creating a plurality of paging tasks with one-to-one correspondence of the plurality of paging data volumes;

通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息。The task message corresponding to the business data of each paging data amount is generated through each paging task corresponding to each paging data amount in the plurality of paging tasks.

可选地，通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息包括：Optionally, generating the task message corresponding to the business data of each paging data amount through each paging task corresponding to each of the paging data amount in the plurality of paging tasks includes:

通过所述每个分页任务向所述业务模块发送查询请求，其中，所述查询请求用于请求查询每个所述分页数据量的业务数据的数据信息；Sending a query request to the service module through each of the paging tasks, wherein the query request is used to request to query data information of the business data of each of the paging data volumes;

接收所述业务模块响应所述查询请求返回的所述数据信息；Receiving the data information returned by the service module in response to the query request;

通过所述每个分页任务使用所述数据信息创建每个所述分页数据量的业务数据中每条数据的原子任务，得到所述每个分页任务所对应的每个所述分页数据量的原子任务，其中，原子任务是用于指示对所述每条数据进行处理的任务；Create an atomic task for each piece of data in the business data of each paging data volume by using the data information for each paging task, and obtain the atomic task for each paging data volume corresponding to each paging task. Tasks, where an atomic task is a task used to instruct to process each piece of data;

通过所述每个分页任务生成每个所述原子任务对应的所述任务消息，其中，所述任务消息用于指示对每个所述原子任务进行处理。The task message corresponding to each of the atomic tasks is generated through each of the paging tasks, where the task message is used to instruct to process each of the atomic tasks.

可选地，将所述任务消息通过消息队列发送至所述业务模块包括：Optionally, sending the task message to the service module through a message queue includes:

将所述任务消息发送至所述消息队列；Sending the task message to the message queue;

通过所述消息队列将所述任务消息转发至所述业务模块。The task message is forwarded to the service module through the message queue.

可选地，通过所述消息队列将所述任务消息转发至所述业务模块包括：Optionally, forwarding the task message to the service module through the message queue includes:

通过所述消息队列在目标时间段内随机将所述任务消息发送至所述业务模块。The task message is randomly sent to the service module within the target time period through the message queue.

可选地，从所述业务模块中查询待所述业务模块处理的业务数据的数据信息包括：Optionally, querying the data information of the business data to be processed by the business module from the business module includes:

每隔目标时间间隔执行定时任务，其中，所述定时任务用于按照所述目标时间间隔驱动任务模块通知所述业务模块处理业务数据；Executing a timing task every target time interval, where the timing task is used to drive a task module according to the target time interval to notify the business module to process business data;

响应所述定时任务向所述业务模块查询所述业务数据的数据总量。In response to the timing task, query the service module for the total amount of data of the service data.

可选地，在将所述任务消息通过消息队列发送至所述业务模块之后，所述方法还包括：Optionally, after sending the task message to the service module through a message queue, the method further includes:

通过所述消息队列检测所述任务消息的发送结果和所述业务模块返回的所述任务消息的处理结果；Detecting the sending result of the task message and the processing result of the task message returned by the service module through the message queue;

在所述发送结果用于指示所述任务消息发送失败，或者，所述处理结果用于指示所述任务消息处理失败的情况下，通过所述消息队列重新向所述业务模块发送所述任务消息。In the case where the sending result is used to indicate that the task message has failed to be sent, or the processing result is used to indicate that the task message processing has failed, the task message is re-sent to the service module through the message queue .

根据本公开实施例的另一方面，还提供了一种业务数据的处理装置，包括：According to another aspect of the embodiments of the present disclosure, there is also provided an apparatus for processing service data, including:

查询模块，设置为从业务模块中查询待所述业务模块处理的业务数据的数据信息；The query module is configured to query the data information of the business data to be processed by the business module from the business module;

生成模块，设置为根据所述数据信息生成与所述业务数据一一对应的任务消息；A generating module, configured to generate a task message one-to-one corresponding to the business data according to the data information;

第一发送模块，设置为将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。The first sending module is configured to send the task message to the service module through a message queue, where the task message is used to instruct the service module to process the service data corresponding to the task message.

根据本公开实施例的另一方面，还提供了一种存储介质，该存储介质包括存储的程序，程序运行时执行上述的方法。According to another aspect of the embodiments of the present disclosure, a storage medium is also provided, the storage medium includes a stored program, and the above-mentioned method is executed when the program runs.

根据本公开实施例的另一方面，还提供了一种电子装置，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，处理器通过计算机程序执行上述的方法。According to another aspect of the embodiments of the present disclosure, there is also provided an electronic device including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the processor executes the above-mentioned method through the computer program.

在本公开实施例中，采用从业务模块中查询待业务模块处理的业务数据的数据信息；根据数据信息生成与业务数据一一对应的任务消息；将任务消息通过消息队列发送至业务模块，其中，任务消息用于指示业务模块对任务消息所对应的业务数据进行处理的方式，在需要通知业务模块对业务数据进行处理时，从业务模块中查询待所述业务模块处理的业务数据的数据信息，根据数据信息生成任务消息，并通过消息队列将任务消息通知给业务模块，避免了业务模块在一定时间内需要集中处理大量业务数据的情况发生，达到了提高服务的高并发和高可用能力的目的，从而实现了提高业务数据的处理效率的技术效果，进而解决了相关技术中业务数据的处理效率较低的技术问题。In the embodiment of the present disclosure, the data information of the business data to be processed by the business module is inquired from the business module; the task message corresponding to the business data one-to-one is generated according to the data information; the task message is sent to the business module through the message queue, wherein The task message is used to instruct the business module to process the business data corresponding to the task message. When the business module needs to be notified to process the business data, the business module queries the data information of the business data to be processed by the business module , Generate task messages based on data information, and notify the business modules of the task messages through the message queue, which avoids the need for the business modules to process a large amount of business data in a certain period of time, and achieves the improvement of high concurrency and high availability of services. The purpose is to achieve the technical effect of improving the processing efficiency of business data, and thereby solve the technical problem of low processing efficiency of business data in related technologies.

附图说明Description of the drawings

此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本发明的实施例，并与说明书一起用于解释本发明的原理。The drawings here are incorporated into the specification and constitute a part of the specification, show embodiments consistent with the present invention, and together with the specification are used to explain the principle of the present invention.

为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can be obtained based on these drawings without creative labor.

图1是根据本公开实施例的业务数据的处理方法的硬件环境的示意图；Fig. 1 is a schematic diagram of a hardware environment of a method for processing business data according to an embodiment of the present disclosure;

图2是根据本公开实施例的一种可选的业务数据的处理方法的流程图；Fig. 2 is a flowchart of an optional method for processing business data according to an embodiment of the present disclosure;

图3是根据本公开可选实施例的一种业务数据的处理过程的示意图；Figure 3 is a schematic diagram of a business data processing process according to an optional embodiment of the present disclosure;

图4是根据本公开实施例的一种可选的业务数据的处理装置的示意图；Fig. 4 is a schematic diagram of an optional service data processing device according to an embodiment of the present disclosure;

图5是根据本公开实施例的一种终端的结构框图。Fig. 5 is a structural block diagram of a terminal according to an embodiment of the present disclosure.

具体实施方式Detailed ways

为了使本技术领域的人员更好地理解本公开方案，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分的实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本公开保护的范围。In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are a part of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of the present disclosure.

需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、***、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms “first”, “second”, etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those clearly listed. Those steps or units may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or equipment.

根据本公开实施例的一方面，提供了一种业务数据的处理的方法实施例。According to an aspect of the embodiments of the present disclosure, an embodiment of a method for processing business data is provided.

可选地，在本实施例中，上述业务数据的处理方法可以应用于如图1所示的由终端101和服务器103所构成的硬件环境中。如图1所示，服务器103通过网络与终端101进行连接，可用于为终端或终端上安装的客户端提供服务(如游戏服务、应用服务等)，可在服务器上或独立于服务器设置数据库，用于为服务器103提供数据存储服务，上述网络包括但不限于：广域网、城域网或局域网，终端101并不限定于PC、手机、平板电脑等。本公开实施例的业务数据的处理方法可以由服务器103来执行，也可以由终端101来执行，还可以是由服务器103和终端101共同执行。其中，终端101执行本公开实施例的业务数据的处理方法也可以是由安装在其上的客户端来执行。Optionally, in this embodiment, the foregoing service data processing method may be applied to the hardware environment formed by the terminal 101 and the server 103 as shown in FIG. 1. As shown in Figure 1, the server 103 is connected to the terminal 101 through the network, and can be used to provide services (such as game services, application services, etc.) for the terminal or the client installed on the terminal. The database can be set on the server or independently of the server. It is used to provide data storage services for the server 103. The aforementioned networks include, but are not limited to: wide area networks, metropolitan area networks, or local area networks. The terminal 101 is not limited to PCs, mobile phones, tablet computers, and the like. The service data processing method of the embodiment of the present disclosure may be executed by the server 103, may also be executed by the terminal 101, or may be executed jointly by the server 103 and the terminal 101. Wherein, the terminal 101 executing the service data processing method of the embodiment of the present disclosure may also be executed by the client installed on it.

图2是根据本公开实施例的一种可选的业务数据的处理方法的流程图，如图2所示，该方法可以包括以下步骤：Fig. 2 is a flowchart of an optional service data processing method according to an embodiment of the present disclosure. As shown in Fig. 2, the method may include the following steps:

步骤S202，从业务模块中查询待所述业务模块处理的业务数据的数据信息；Step S202, query data information of the business data to be processed by the business module from the business module;

步骤S204，根据所述数据信息生成与所述业务数据一一对应的任务消息；Step S204, generating task messages corresponding to the service data one-to-one according to the data information;

步骤S206，将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。Step S206: Send the task message to the service module through a message queue, where the task message is used to instruct the service module to process the service data corresponding to the task message.

通过上述步骤S202至步骤S206，在需要通知业务模块对业务数据进行处理时，从业务模块中查询待所述业务模块处理的业务数据的数据信息，根据数据信息生成任务消息，并通过消息队列将任务消息通知给业务模块，避免了业务模块在一定时间内需要集中处理大量业务数据的情况发生，达到了提高服务的高并发和高可用能力的目的，从而实现了提高业务数据的处理效率的技术效果，进而解决了相关技术中业务数据的处理效率较低的技术问题。Through the above steps S202 to S206, when the business module needs to be notified to process the business data, the business module is inquired about the data information of the business data to be processed by the business module, the task message is generated according to the data information, and the message queue The task message is notified to the business module, which avoids the situation that the business module needs to process a large amount of business data in a certain period of time, achieves the purpose of improving the high concurrency and high availability of the service, thereby realizing the technology to improve the processing efficiency of business data Effect, and thus solve the technical problem of low processing efficiency of business data in related technologies.

可选地，在本实施例中，上述业务数据的处理方法可以但不限于由任务模块(又可以称为任务***)来执行。上述业务模块也可以称为业务***。上述业务数据的处理方法可以但不限于应用于需要任务模块通知业务模块处理业务数据的场景中。Optionally, in this embodiment, the foregoing business data processing method may be, but not limited to, executed by a task module (also referred to as a task system). The above-mentioned service module may also be called a service system. The foregoing business data processing method can be, but is not limited to, applied to scenarios that require the task module to notify the business module to process the business data.

在步骤S202提供的技术方案中，上述任务模块和业务模块可以但不限于是各种功能的应用中的任务模块和业务模块，比如：在线教育应用、即时通讯应用、社区空间应用、游戏应用、购物应用、浏览器应用、金融应用、多媒体应用、直播应用等。In the technical solution provided in step S202, the above-mentioned task modules and business modules may be, but are not limited to, task modules and business modules in various functional applications, such as online education applications, instant messaging applications, community space applications, game applications, Shopping applications, browser applications, financial applications, multimedia applications, live broadcast applications, etc.

作为一种可选的实施例，所述数据信息包括所述业务数据的数据总量，其中，根据所述数据信息生成与所述业务数据一一对应的所述任务消息包括：As an optional embodiment, the data information includes the total amount of data of the service data, and generating the task message corresponding to the service data one-to-one according to the data information includes:

S11，将所述数据总量划分为多个分页数据量；S11. Divide the total amount of data into multiple paging data amounts;

S12，创建所述多个分页数据量一一对应的多个分页任务；S12, creating a plurality of paging tasks corresponding to the plurality of paging data amounts in a one-to-one manner;

S13，通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息。S13: Generate the task message corresponding to the business data of each paging data amount through each paging task corresponding to each paging data amount in the plurality of paging tasks.

可选地，在本实施例中，数据信息可以但不限于包括业务数据的数据总量，也就是说，获取的数据信息可以是业务模块需要处理的数据量。再根据数据量进行分页，分页方式不限，比如可以平均划分，也可以按照一定比例划分，或者还可以随机划分，也可以通过训练模型得到一个能够自动进行数量划分的AI模型来进行分页数据量的划分。Optionally, in this embodiment, the data information may, but is not limited to, include the total amount of business data, that is, the acquired data information may be the amount of data that the business module needs to process. Paging is performed according to the amount of data. The paging method is not limited. For example, it can be divided equally, divided according to a certain proportion, or divided randomly, or an AI model that can automatically divide the number can be obtained by training the model to divide the amount of data. The division.

比如：查询到10000条数据，划分为10个分页数据量，则每个分页数据量为1000条数据。For example, if 10,000 pieces of data are queried and divided into 10 pieces of data, each piece of data is 1,000 pieces of data.

可选地，在本实施例中，为多个分页数据量中每个分页数据量创建一个分页任务来处理该分页数据量。每个分页任务用于生成该分页任务负责的分页数据量的业务数据对应的任务消息。Optionally, in this embodiment, a paging task is created for each paging data amount in the multiple paging data amounts to process the paging data amount. Each paging task is used to generate a task message corresponding to the business data of the paging data volume that the paging task is responsible for.

作为一种可选的实施例，通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息包括：As an optional embodiment, generating the task message corresponding to the business data of each paging data amount through each paging task corresponding to each paging data amount of the plurality of paging tasks includes:

S21，通过所述每个分页任务向所述业务模块发送查询请求，其中，所述查询请求用于请求查询每个所述分页数据量的业务数据的数据信息；S21, sending a query request to the service module through each of the paging tasks, where the query request is used to request to query data information of the business data of each of the paging data volumes;

S22，接收所述业务模块响应所述查询请求返回的所述数据信息；S22: Receive the data information returned by the service module in response to the query request;

S23，通过所述每个分页任务使用所述数据信息创建每个所述分页数据量的业务数据中每条数据的原子任务，得到所述每个分页任务所对应的每个所述分页数据量的原子任务，其中，原子任务是用于指示对所述每条数据进行处理的任务；S23. Use the data information to create an atomic task for each piece of data in the business data of each paging data volume through each paging task, to obtain each paging data volume corresponding to each paging task The atomic task of, where the atomic task is a task used to instruct the processing of each piece of data;

S24，通过所述每个分页任务生成每个所述原子任务对应的所述任务消息，其中，所述任务消息用于指示对每个所述原子任务进行处理。S24. Generate the task message corresponding to each of the atomic tasks through each of the paging tasks, where the task message is used to instruct to process each of the atomic tasks.

可选地，在本实施例中，每个分页任务向业务模块查询其负责的每个数据的明细(即数据信息)。使用数据信息来生成原子任务。并生成用于通知消息队列的任务消息。Optionally, in this embodiment, each paging task queries the business module for the details (ie data information) of each data for which it is responsible. Use data information to generate atomic tasks. And generate a task message used to notify the message queue.

在上述实施例中，对于生成的10个分页任务，每一个分页任务会去查询任务的明细，如每一个分页任务查询1000条，生成1000个原子任务，十个分页任务就生成10*1000个原子任务，最终一共也是10000个原子任务。In the above embodiment, for the generated 10 paging tasks, each paging task will query the details of the task. For example, each paging task queries 1000, generates 1000 atomic tasks, and 10 paging tasks generates 10*1000. Atomic tasks, and ultimately a total of 10,000 atomic tasks.

作为一种可选的实施例，将所述任务消息通过消息队列发送至所述业务模块包括：As an optional embodiment, sending the task message to the service module through a message queue includes:

S31，将所述任务消息发送至所述消息队列；S31: Send the task message to the message queue;

S32，通过所述消息队列将所述任务消息转发至所述业务模块。S32. Forward the task message to the service module through the message queue.

可选地，在本实施例中，可以由消息队列实现对任务消息的转发。Optionally, in this embodiment, the task message can be forwarded by the message queue.

可选地，在本实施例中，消息队列可以单不限于采用MQ(Message Queue，消息队列)机制，MQ是一种分布式异步消息队列，通过异步通知的方式，达到削峰和限流的作用。发送方通过将请求数据json(JavaScript Object Notation，JS对象简谱)化并持久化到MQ队列中，MQ中间件再将队列数据发送给订阅消息的接收方，接收方收到通知后，解析队列数据进行业务处理。Optionally, in this embodiment, the message queue may not be limited to adopting the MQ (Message Queue, message queue) mechanism. MQ is a distributed asynchronous message queue that achieves peak cutting and current limiting through asynchronous notification. effect. The sender json (JavaScript Object Notation, JS object notation) of the request data is made and persisted in the MQ queue. The MQ middleware then sends the queue data to the receiver who subscribes to the message. After the receiver receives the notification, it parses the queue data. Perform business processing.

采用MQ机制在本实施例中的优势其一是异步化，可以保证任务***高吞吐，无需同步等待业务***处理结果，快速响应批量任务处理。其二是MQ的自动重试机制能够保证业务***实例在处理异步请求时出现异常可以自动重试。One of the advantages of adopting the MQ mechanism in this embodiment is asynchronous, which can ensure high throughput of the task system, does not need to wait for the processing result of the business system synchronously, and quickly responds to batch task processing. The second is that MQ's automatic retry mechanism can ensure that the business system instance can automatically retry when an exception occurs when processing asynchronous requests.

可选地，在本实施例中，单任务处理数据时，由同步调用业务***调整为消息通知，使得任务线程执行效率更高，提升任务处理吞吐量，由消息保证重试，同时解决业务***单点问题。Optionally, in this embodiment, when a single task is processing data, the synchronous invocation of the business system is adjusted to the message notification, which makes the task thread execution more efficient, improves the task processing throughput, guarantees the retry by the message, and solves the business system at the same time. Single point of issue.

作为一种可选的实施例，通过所述消息队列将所述任务消息转发至所述业务模块包括：As an optional embodiment, forwarding the task message to the service module through the message queue includes:

S41，通过所述消息队列在目标时间段内随机将所述任务消息发送至所述业务模块。S41: Randomly send the task message to the service module within a target time period through the message queue.

可选地，在本实施例中，可以对消息发送进行随机处理，让业务模块在一定时间段内收到1万个消息通知，然后去处理。使用分布式任务和随机时间的方式去处理，把以前的集中时间处理弱化为分时处理，避免在短时间内造成并发问题。Optionally, in this embodiment, the message sending can be randomly processed, so that the service module receives 10,000 message notifications within a certain period of time, and then processes them. Use distributed tasks and random time to process, weaken the previous centralized time processing into time-sharing processing, and avoid concurrency problems in a short time.

可选地，在本实施例中，任务***查询业务***数据总量，根据总量分页切分，使用分布式任务与随机时间处理结合的方式，生成分布式分页任务及对应原子任务，提升任务并发处理能力，同时通过随机时间有效控制业务***并发处理量。Optionally, in this embodiment, the task system queries the total amount of business system data, paging and segmenting according to the total amount, and using a combination of distributed tasks and random time processing to generate distributed paging tasks and corresponding atomic tasks to improve tasks Concurrent processing capability, while effectively controlling the concurrent processing volume of the business system through random time.

作为一种可选的实施例，从所述业务模块中查询待所述业务模块处理的业务数据的数据信息包括：As an optional embodiment, querying data information of the business data to be processed by the business module from the business module includes:

S51，每隔目标时间间隔执行定时任务，其中，所述定时任务用于按照所述目标时间间隔驱动任务模块通知所述业务模块处理业务数据；S51: Execute a timing task every target time interval, where the timing task is used to drive a task module according to the target time interval to notify the business module to process business data;

S52，响应所述定时任务向所述业务模块查询所述业务数据的数据总量。S52: In response to the timing task, query the service module for the total amount of data of the service data.

可选地，在本实施例中，任务模块在处理任务的时候，可以由定时任务去驱动任务模块通知业务模块处理业务数据，定时任务执行的目标时间间隔可以但不限于为一周，一个月，一个季度，半年，一年等等。Optionally, in this embodiment, when the task module is processing the task, the task module can be driven by the timed task to notify the business module to process the business data. The target time interval for the execution of the timed task can be, but not limited to, one week or one month. One quarter, six months, one year, etc.

作为一种可选的实施例，在将所述任务消息通过消息队列发送至所述业务模块之后，所述方法还包括：As an optional embodiment, after sending the task message to the service module through a message queue, the method further includes:

S61，通过所述消息队列检测所述任务消息的发送结果和所述业务模块返回的所述任务消息的处理结果；S61: Detect the sending result of the task message and the processing result of the task message returned by the service module through the message queue;

S62，在所述发送结果用于指示所述任务消息发送失败，或者，所述处理结果用于指示所述任务消息处理失败的情况下，通过所述消息队列重新向所述业务模块发送所述任务消息。S62. In a case where the sending result is used to indicate that the task message has failed to be sent, or the processing result is used to indicate that the task message has failed to be processed, re-send the message to the service module through the message queue. Mission message.

可选地，在本实施例中，通过消息队列监控任务消息是否发送成功或者是否处理成功，如果没有发送成功或者没有处理成功，则可以触发消息队列的重试机制重新发送任务消息，从而通过重试机制可以大大提高任务数据的处理成功率。Optionally, in this embodiment, the message queue is used to monitor whether the task message is successfully sent or processed. If the message is not sent or processed successfully, the retry mechanism of the message queue can be triggered to resend the task message, thereby retransmitting the task message. The trial mechanism can greatly improve the success rate of task data processing.

本公开还提供了一种可选实施例，该可选实施例提供了一种任务***通知业务***处理业务数据的方式，图3是根据本公开可选实施例的一种业务数据的处理过程的示意图，如图3所示，有两个服务，一个是任务服务(相当于上述任务模块)，一个是基本法服务，基本法服务相当于是一个业务服务(相当于上述业务模块)。任务服务在处理任务的时候，由定时任务去驱动，定时生成一个初始任务，这个初始任务可通过MQ消息的形式去通知，发送消息过后会去查询业务***的数据总数。比如，查出来是1万条数据，就可以通过分片去申请一个分页的分布式任务，假设1万条数据按1000去分，就分成了十个分页任务。对于这十个分页任务，每一个分页任务会去查询任务的明细，如每一个分页任务查询1000条，生成1000个原子任务，10个分页任务就生成10*1000个原子任务，最终一共也是一万个原子任务。原子任务在执行的时候，通过MQ的形式通知业务***，业务***收到这个消息过后，进行任务的处理，***可以对消息发送进行随机处理，让业务***在一定时间段内收到1万个消息通知，然后去处理。The present disclosure also provides an optional embodiment. The optional embodiment provides a way for the task system to notify the business system to process business data. FIG. 3 is a business data processing procedure according to an optional embodiment of the present disclosure. As shown in Figure 3, there are two services, one is a task service (equivalent to the above task module), one is a basic law service, and the basic law service is equivalent to a business service (equivalent to the above business module). When the task service is processing tasks, it is driven by a timed task to generate an initial task regularly. This initial task can be notified in the form of an MQ message. After the message is sent, it will query the total number of data in the business system. For example, if it is found that there are 10,000 pieces of data, you can apply for a paging distributed task through sharding. Assuming that 10,000 pieces of data are divided into 1000, it is divided into ten paging tasks. For these ten paging tasks, each paging task will query the details of the task. For example, each paging task will query 1000, generating 1000 atomic tasks, and 10 paging tasks will generate 10*1000 atomic tasks, and the final total is also one. Ten thousand atomic tasks. When the atomic task is being executed, the business system is notified in the form of MQ. After the business system receives the message, it processes the task. The system can randomly process the message sending, so that the business system can receive 10,000 within a certain period of time. The message is notified and then processed.

对于大部分的任务，定时驱动任务是由任务***定时去发一个消息通知业务***，由业务***就自己去任务；业务***可以采用分片的机制来完成任务，即在通知的时候，业务***会根据自己的机器的分片情况，去分片执行，直到所有数据处理完成。但是这种方案存在一个问题，就是***在任务执行的时候，某个执行时间点的并发压力会非常大，会影响其正常的业务服务，而且通过分片技术去执行时，有可能单机器的服务器没办法保证数据最终执行正确，有可能处理失败或者服务挂掉。这就需要异常的监控和重试机制来保证最终的正确，这种设计的方式就会比较复杂。For most tasks, the timing-driven task is that the task system periodically sends a message to notify the business system, and the business system does the task by itself; the business system can use the fragmentation mechanism to complete the task, that is, when the notification is made, the business system According to the fragmentation situation of its own machine, it will perform fragmentation until all data processing is completed. However, there is a problem with this solution. When the system is executing tasks, the concurrency pressure at a certain execution time point will be very large, which will affect its normal business services. And when it is executed through sharding technology, it may be single-machine. The server has no way to ensure that the data is finally executed correctly, and it may fail to process or the service will hang. This requires an abnormal monitoring and retry mechanism to ensure the final correctness, and this design method will be more complicated.

本可选实施例中，采用了消息驱动的分页模式的任务处理方式。任务***在去做定时触发任务的时候不是直接告诉业务***需要处理的数据，而是首先去查询业务***，查询它需要处理的数据总量，然后根据数据总量去做切分，把总的大数据量切分成N片小的数据量。对于切分后的数据，使用分布式任务和随机时间的方式去处理，把以前的集中时间处理弱化为分时处理，避免在短时间内造成并发问题。In this optional embodiment, a message-driven paging mode task processing mode is adopted. When the task system does timing trigger tasks, it does not directly tell the business system the data that needs to be processed. Instead, it first queries the business system, queries the total amount of data it needs to process, and then divides it according to the total amount of data to divide the total The large data volume is divided into N pieces of small data volume. For the segmented data, use distributed tasks and random time to process, weaken the previous centralized time processing into time-sharing processing, and avoid concurrency problems in a short time.

对于每个分页任务，它会生成自身的一个分布式的分页任务和对应的原子任务。比如100万的数据，可以按照1万条数据为单位分页，分成100个分页任务。这样，每个分页任务只需要关心自己所在那一页的1万条数据，可以提升任务并发处理能力，这100万的数据不是串行去处理，也不是按业务***的分片去处理，而是由任务***先把这100万的数据分成100个分页任务，每一个分页任务去处理1万条数据。For each paging task, it will generate its own distributed paging task and the corresponding atomic task. For example, 1 million data can be paged according to 10,000 pieces of data and divided into 100 paging tasks. In this way, each paging task only needs to care about the 10,000 pieces of data on the page it is on, which can improve the concurrent processing capacity of the task. This 1 million data is not processed serially, nor is it processed according to the fragmentation of the business system. The task system first divides the 1 million data into 100 paging tasks, and each paging task processes 10,000 pieces of data.

在这个分页的基础上，可再加上消息驱动，对于分页任务生成的1万个原子任务，每一个原子任务在处理数据的时候，不是同步去调业务***，因为同步调业务***时耗时较长，假设一个原子任务的数据处理的时长是一分钟，那每次调用就需要维持一分钟，当原子任务的数量较大时这个调用时长就会很长，有可能会导致业务***的链接数被耗尽。本可选实施例中采用的是通过消息去处理的方式，每个原子任务在执行的时候，只需要发一个消息通知，通过消息驱动业务***，业务***监听到这个消息去做处理，相当于是把一个同步过程改为消息通知的异步方式。这种方式可以让任务***的任务线程的执行效率变得更高，因为相对于维持一分钟的调用，发送一个消息的耗时是极短的，这样任务***就可以用节约的时间继续发送别的消息。On the basis of this paging, message-driven can be added. For the 10,000 atomic tasks generated by the paging task, when each atomic task is processing data, it is not synchronized to adjust the business system, because it takes time to synchronize the business system. Longer, assuming that the data processing duration of an atomic task is one minute, then each call needs to be maintained for one minute. When the number of atomic tasks is large, the call duration will be very long, which may cause the link of the business system The number is exhausted. In this optional embodiment, the method of processing through messages is adopted. When each atomic task is executed, only one message notification needs to be sent. Through the message-driven business system, the business system listens to the message to perform processing, which is equivalent to Change a synchronous process to an asynchronous way of message notification. In this way, the execution efficiency of the task thread of the task system becomes higher, because compared to the one-minute call, the time to send a message is extremely short, so that the task system can continue to send other messages with the saved time News.

同时，通过消息的方式也能让各个业务***的重试失败的任务，因为如果有机器在听到一个消息过后，有可能这个消息是这个机器本身去消费的、也可能是其他机器去消费的，如果这个机器有异常的没法提供服务，这个消息也会由其他机器去处理，去消费。就不会出现单点问题。At the same time, the way of messages can also allow various business systems to retry failed tasks, because if a machine hears a message, it is possible that the message is consumed by the machine itself, or it may be consumed by other machines. If this machine is abnormal and cannot provide services, the message will also be processed by other machines for consumption. There will be no single point problem.

需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本公开并不受所描述的动作顺序的限制，因为依据本公开，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于可选实施例，所涉及的动作和模块并不一定是本公开所必须的。It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the described sequence of actions. Because according to the present disclosure, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本公开各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is Better implementation. Based on this understanding, the technical solution of the present disclosure essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present disclosure.

根据本公开实施例的另一个方面，还提供了一种用于实施上述业务数据的处理方法的业务数据的处理装置。图4是根据本公开实施例的一种可选的业务数据的处理装置的示意图，如图4所示，该装置可以包括：According to another aspect of the embodiments of the present disclosure, there is also provided a service data processing device for implementing the above-mentioned service data processing method. Fig. 4 is a schematic diagram of an optional service data processing device according to an embodiment of the present disclosure. As shown in Fig. 4, the device may include:

查询模块42，设置为从业务模块中查询待所述业务模块处理的业务数据的数据信息；The query module 42 is configured to query the data information of the business data to be processed by the business module from the business module;

生成模块44，设置为根据所述数据信息生成与所述业务数据一一对应的任务消息；A generating module 44, configured to generate task messages corresponding to the business data one-to-one according to the data information;

第一发送模块46，设置为将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。The first sending module 46 is configured to send the task message to the service module through a message queue, wherein the task message is used to instruct the service module to process the service data corresponding to the task message .

需要说明的是，该实施例中的查询模块42可以设置为执行本公开实施例中的步骤S202，该实施例中的生成模块44可以设置为执行本公开实施例中的步骤S204，该实施例中的第一发送模块46可以设置为执行本公开实施例中的步骤S206。It should be noted that the query module 42 in this embodiment can be configured to perform step S202 in the embodiment of the present disclosure, and the generation module 44 in this embodiment can be configured to perform step S204 in the embodiment of the present disclosure. The first sending module 46 in can be configured to execute step S206 in the embodiment of the present disclosure.

此处需要说明的是，上述模块与对应的步骤所实现的示例和应用场景相同，但不限于上述实施例所公开的内容。需要说明的是，上述模块作为装置的一部分可以运行在如图1所示的硬件环境中，可以通过软件实现，也可以通过硬件实现。It should be noted here that the examples and application scenarios implemented by the foregoing modules and corresponding steps are the same, but are not limited to the content disclosed in the foregoing embodiments. It should be noted that, as a part of the device, the above-mentioned modules can run in the hardware environment as shown in FIG. 1, and can be implemented by software or hardware.

通过上述模块，在需要通知业务模块对业务数据进行处理时，从业务模块中查询待所述业务模块处理的业务数据的数据信息，根据数据信息生成任务消息，并通过消息队列将任务消息通知给业务模块，避免了业务模块在一定时间内需要集中处理大量业务数据的情况发生，达到了提高服务的高并发和高可用能力的目的，从而实现了提高业务数据的处理效率的技术效果，进而解决了相关技术中业务数据的处理效率较低的技术问题。Through the above-mentioned module, when the business module needs to be notified to process the business data, the business module is inquired about the data information of the business data to be processed by the business module, the task message is generated according to the data information, and the task message is notified through the message queue The business module avoids the situation that the business module needs to process a large amount of business data in a certain period of time, and achieves the purpose of improving the high concurrency and high availability of services, thereby achieving the technical effect of improving the processing efficiency of business data, and then solving This solves the technical problem of low processing efficiency of business data in related technologies.

作为一种可选的实施例，所述数据信息包括所述业务数据的数据总量，其中，所述生成模块包括：As an optional embodiment, the data information includes the total amount of data of the business data, wherein the generating module includes:

划分单元，设置为将所述数据总量划分为多个分页数据量；A dividing unit, configured to divide the total amount of data into multiple paging data amounts;

创建单元，设置为创建所述多个分页数据量一一对应的多个分页任务；A creation unit, configured to create a plurality of paging tasks corresponding to the plurality of paging data volumes one-to-one;

生成单元，设置为通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息。The generating unit is configured to generate the task message corresponding to the business data of each paging data amount through each paging task corresponding to each paging data amount in the plurality of paging tasks.

作为一种可选的实施例，所述生成单元设置为：As an optional embodiment, the generating unit is set to:

通过所述每个分页任务向所述业务模块发送查询请求，其中，所述查询请求用于请求查询每个所述分页数据量的业务数据的数据信息；Sending a query request to the service module through each of the paging tasks, where the query request is used to request to query data information of the business data of each of the paging data volumes;

作为一种可选的实施例，所述第一发送模块包括：As an optional embodiment, the first sending module includes:

发送单元，设置为将所述任务消息发送至所述消息队列；A sending unit, configured to send the task message to the message queue;

转发单元，设置为通过所述消息队列将所述任务消息转发至所述业务模块。The forwarding unit is configured to forward the task message to the service module through the message queue.

作为一种可选的实施例，所述转发单元设置为：As an optional embodiment, the forwarding unit is set to:

作为一种可选的实施例，所述查询模块设置为：As an optional embodiment, the query module is set to:

作为一种可选的实施例，所述装置还包括：As an optional embodiment, the device further includes:

检测模块，设置为在将所述任务消息通过消息队列发送至所述业务模块之后，通过所述消息队列检测所述任务消息的发送结果和所述业务模块返回的所述任务消息的处理结果；A detection module, configured to detect the sending result of the task message and the processing result of the task message returned by the service module through the message queue after the task message is sent to the service module through the message queue;

第二发送模块，设置为在所述发送结果用于指示所述任务消息发送失败，或者，所述处理结果用于指示所述任务消息处理失败的情况下，通过所述消息队列重新向所述业务模块发送所述任务消息。The second sending module is configured to retransmit to the message queue through the message queue when the sending result is used to indicate that the task message has failed to be sent, or the processing result is used to indicate that the task message has failed to be sent. The service module sends the task message.

此处需要说明的是，上述模块与对应的步骤所实现的示例和应用场景相同，但不限于上述实施例所公开的内容。需要说明的是，上述模块作为装置的一部分可以运行在如图1所示的硬件环境中，可以通过软件实现，也可以通过硬件实现，其中，硬件环境包括网络环境。It should be noted here that the examples and application scenarios implemented by the foregoing modules and corresponding steps are the same, but are not limited to the content disclosed in the foregoing embodiments. It should be noted that, as a part of the device, the above-mentioned modules can run in the hardware environment as shown in FIG. 1, and can be implemented by software or hardware, where the hardware environment includes a network environment.

根据本公开实施例的另一个方面，还提供了一种用于实施上述业务数据的处理方法的服务器或终端。According to another aspect of the embodiments of the present disclosure, there is also provided a server or terminal for implementing the foregoing method for processing service data.

图5是根据本公开实施例的一种终端的结构框图，如图5所示，该终端可以包括：一个或多个(图中仅示出一个)处理器501、存储器503、以及传输装置505，如图5所示，该终端还可以包括输入输出设备507。FIG. 5 is a structural block diagram of a terminal according to an embodiment of the present disclosure. As shown in FIG. 5, the terminal may include: one or more (only one is shown in the figure) processor 501, memory 503, and transmission device 505 As shown in FIG. 5, the terminal may also include an input and output device 507.

其中，存储器503可设置为存储软件程序以及模块，如本公开实施例中的业务数据的处理方法和装置对应的程序指令/模块，处理器501通过运行存储在存储器503内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的业务数据的处理方法。存储器503可包括高速随机存储器，还可以包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器503可进一步包括相对于处理器501远程设置的存储器，这些远程存储器可以通过网络连接至终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 503 can be configured to store software programs and modules, such as the program instructions/modules corresponding to the business data processing method and device in the embodiments of the present disclosure. The processor 501 runs the software programs and modules stored in the memory 503, In this way, various functional applications and data processing are executed, that is, the above-mentioned business data processing method is realized. The memory 503 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 503 may further include a memory remotely provided with respect to the processor 501, and these remote memories may be connected to the terminal through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

上述的传输装置505设置为经由一个网络接收或者发送数据，还可以设置为处理器与存储器之间的数据传输。上述的网络可选实例可包括有线网络及无线网络。在一个实例中，传输装置505包括一个网络适配器(Network Interface Controller，NIC)，其可通过网线与其他网络设备与路由器相连从而可与互联网或局域网进行通讯。在一个实例中，传输装置505为射频(Radio Frequency，RF)模块，其设置为通过无线方式与互联网进行通讯。The aforementioned transmission device 505 is configured to receive or send data via a network, and may also be configured to transmit data between the processor and the memory. The above-mentioned optional examples of networks may include wired networks and wireless networks. In one example, the transmission device 505 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 505 is a radio frequency (RF) module, which is configured to communicate with the Internet in a wireless manner.

其中，可选地，存储器503设置为存储应用程序。Wherein, optionally, the memory 503 is configured to store an application program.

处理器501可以通过传输装置505调用存储器503存储的应用程序，以执行下述步骤：The processor 501 may call the application program stored in the memory 503 through the transmission device 505 to perform the following steps:

S1，从业务模块中查询待所述业务模块处理的业务数据的数据信息；S1: Query data information of the business data to be processed by the business module from the business module;

S2，根据所述数据信息生成与所述业务数据一一对应的任务消息；S2, generating task messages corresponding to the business data one-to-one according to the data information;

S3，将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。S3. Send the task message to the service module through a message queue, where the task message is used to instruct the service module to process the service data corresponding to the task message.

采用本公开实施例，提供了一种业务数据的处理的方案。在需要通知业务模块对业务数据进行处理时，从业务模块中查询待所述业务模块处理的业务数据的数据信息，根据数据信息生成任务消息，并通过消息队列将任务消息通知给业务模块，避免了业务模块在一定时间内需要集中处理大量业务数据的情况发生，达到了提高服务的高并发和高可用能力的目的，从而实现了提高业务数据的处理效率的技术效果，进而解决了相关技术中业务数据的处理效率较低的技术问题。By adopting the embodiments of the present disclosure, a solution for processing business data is provided. When it is necessary to notify the business module to process the business data, query the data information of the business data to be processed by the business module from the business module, generate task messages based on the data information, and notify the task messages to the business modules through the message queue to avoid The situation that the business module needs to process a large amount of business data in a certain period of time occurs, and the purpose of improving the high concurrency and high availability of the service is achieved, thereby achieving the technical effect of improving the processing efficiency of business data, thereby solving the problem of related technologies. Technical problem of low processing efficiency of business data.

可选地，本实施例中的可选示例可以参考上述实施例中所描述的示例，本实施例在此不再赘述。Optionally, for optional examples in this embodiment, reference may be made to the examples described in the foregoing embodiment, and this embodiment will not be repeated here.

本领域普通技术人员可以理解，图5所示的结构仅为示意，终端可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌上电脑以及移动互联网设备(Mobile Internet Devices，MID)、PAD等终端设备。图5其并不对上述电子装置的结构造成限定。例如，终端还可包括比图5中所示更多或者更少的组件(如网络接口、显示装置等)，或者具有与图5所示不同的配置。Those of ordinary skill in the art can understand that the structure shown in Figure 5 is only for illustration, and the terminal can be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and a mobile Internet Device (MID), Terminal equipment such as PAD. FIG. 5 does not limit the structure of the above-mentioned electronic device. For example, the terminal may also include more or fewer components (such as a network interface, a display device, etc.) than shown in FIG. 5, or have a different configuration from that shown in FIG. 5.

本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the relevant hardware of the terminal device through a program. The program can be stored in a computer-readable storage medium, and the storage medium can be Including: flash disk, read-only memory (Read-Only Memory, ROM), random access device (Random Access Memory, RAM), magnetic disk or optical disk, etc.

本公开的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以设置为执行业务数据的处理方法的程序代码。The embodiment of the present disclosure also provides a storage medium. Optionally, in this embodiment, the above-mentioned storage medium may be configured as program code for executing the method for processing service data.

可选地，在本实施例中，上述存储介质可以位于上述实施例所示的网络中的多个网络设备中的至少一个网络设备上。Optionally, in this embodiment, the foregoing storage medium may be located on at least one of the multiple network devices in the network shown in the foregoing embodiment.

可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps:

可选地，本实施例中的可选示例可以参考上述实施例中所描述的示例，本实施例在此不再赘述。Optionally, for the optional examples in this embodiment, reference may be made to the examples described in the foregoing embodiment, and this embodiment will not be repeated here.

可选地，在本实施例中，上述存储介质可以包括但不限于：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。Optionally, in this embodiment, the foregoing storage medium may include, but is not limited to: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk Various media that can store program codes, such as discs or optical discs.

上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。The serial numbers of the above-mentioned embodiments of the present disclosure are only for description, and do not represent the superiority or inferiority of the embodiments.

上述实施例中的集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在上述计算机可读取的存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在存储介质中，包括若干指令用以使得一台或多台计算机设备(可为个人计算机、服务器或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。If the integrated unit in the foregoing embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in the foregoing computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, A number of instructions are included to enable one or more computer devices (which may be personal computers, servers, or network devices, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present disclosure.

在本公开的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。In the above-mentioned embodiments of the present disclosure, the description of each embodiment has its own focus. For a part that is not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

在本公开所提供的几个实施例中，应该理解到，所揭露的客户端，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。In the several embodiments provided in the present disclosure, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, units or modules, and may be in electrical or other forms.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

以上所述仅是本公开的可选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本公开原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本公开的保护范围。The above are only optional implementations of the present disclosure. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of the present disclosure, several improvements and modifications can be made. These improvements and modifications It should also be regarded as the protection scope of the present disclosure.

Claims

一种业务数据的处理方法，包括：A method for processing business data, including:

从业务模块中查询待所述业务模块处理的业务数据的数据信息；Query data information of the business data to be processed by the business module from the business module;

根据所述数据信息生成与所述业务数据一一对应的任务消息；Generating task messages corresponding to the business data one-to-one according to the data information;

将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。The task message is sent to the service module through a message queue, where the task message is used to instruct the service module to process the service data corresponding to the task message.
根据权利要求1所述的方法，其中，所述数据信息包括所述业务数据的数据总量，其中，根据所述数据信息生成与所述业务数据一一对应的所述任务消息包括：The method according to claim 1, wherein the data information includes the total amount of data of the service data, and generating the task message corresponding to the service data one-to-one according to the data information comprises:

将所述数据总量划分为多个分页数据量；Dividing the total amount of data into multiple paging data amounts;

创建所述多个分页数据量一一对应的多个分页任务；Creating a plurality of paging tasks with one-to-one correspondence of the plurality of paging data volumes;

通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息。The task message corresponding to the business data of each paging data amount is generated through each paging task corresponding to each paging data amount in the plurality of paging tasks.
根据权利要求2所述的方法，其中，通过所述多个分页任务中每个所述分页数据量对应的每个分页任务生成每个所述分页数据量的业务数据对应的所述任务消息包括：The method according to claim 2, wherein generating the task message corresponding to the business data of each of the paging data by using each paging task corresponding to each of the paging data of the plurality of paging tasks comprises :

通过所述每个分页任务向所述业务模块发送查询请求，其中，所述查询请求用于请求查询每个所述分页数据量的业务数据的数据信息；Sending a query request to the service module through each of the paging tasks, wherein the query request is used to request to query data information of the business data of each of the paging data volumes;

接收所述业务模块响应所述查询请求返回的所述数据信息；Receiving the data information returned by the service module in response to the query request;

通过所述每个分页任务使用所述数据信息创建每个所述分页数据量的业务数据中每条数据的原子任务，得到所述每个分页任务所对应的每个所述分页数据量的原子任务，其中，原子任务是用于指示对所述每条数据进行处理的任务；Using the data information to create an atomic task for each piece of data in the business data of each paging task by using the data information for each paging task, to obtain the atomic task of each paging data corresponding to each paging task Tasks, where an atomic task is a task used to instruct to process each piece of data;

通过所述每个分页任务生成每个所述原子任务对应的所述任务消息，其中，所述任务消息用于指示对每个所述原子任务进行处理。The task message corresponding to each of the atomic tasks is generated through each of the paging tasks, where the task message is used to instruct to process each of the atomic tasks.
根据权利要求1所述的方法，其中，将所述任务消息通过消息队列发送至所述业务模块包括：The method according to claim 1, wherein sending the task message to the service module through a message queue comprises:

将所述任务消息发送至所述消息队列；Sending the task message to the message queue;

通过所述消息队列将所述任务消息转发至所述业务模块。The task message is forwarded to the service module through the message queue.
根据权利要求4所述的方法，其中，通过所述消息队列将所述任务消息转发至所述业务模块包括：The method according to claim 4, wherein forwarding the task message to the service module through the message queue comprises:

通过所述消息队列在目标时间段内随机将所述任务消息发送至所述业务模块。The task message is randomly sent to the service module within the target time period through the message queue.
根据权利要求1所述的方法，其中，从所述业务模块中查询待所述业务模块处理的业务数据的数据信息包括：The method according to claim 1, wherein querying data information of the business data to be processed by the business module from the business module comprises:

每隔目标时间间隔执行定时任务，其中，所述定时任务用于按照所述目标时间间隔驱动任务模块通知所述业务模块处理业务数据；Executing a timing task every target time interval, where the timing task is used to drive a task module according to the target time interval to notify the business module to process business data;

响应所述定时任务向所述业务模块查询所述业务数据的数据总量。In response to the timing task, query the service module for the total amount of data of the service data.
根据权利要求1所述的方法，其中，在将所述任务消息通过消息队列发送至所述业务模块之后，所述方法还包括：The method according to claim 1, wherein after sending the task message to the service module through a message queue, the method further comprises:

通过所述消息队列检测所述任务消息的发送结果和所述业务模块返回的所述任务消息的处理结果；Detecting the sending result of the task message and the processing result of the task message returned by the service module through the message queue;

在所述发送结果用于指示所述任务消息发送失败，或者，所述处理结果用于指示所述任务消息处理失败的情况下，通过所述消息队列重新向所述业务模块发送所述任务消息。In the case where the sending result is used to indicate that the task message has failed to be sent, or the processing result is used to indicate that the task message processing has failed, the task message is re-sent to the service module through the message queue .
一种业务数据的处理装置，包括：A processing device for business data, including:

查询模块，设置为从业务模块中查询待所述业务模块处理的业务数据的数据信息；The query module is configured to query the data information of the business data to be processed by the business module from the business module;

生成模块，设置为根据所述数据信息生成与所述业务数据一一对应的任务消息；A generating module, configured to generate task messages corresponding to the business data one-to-one according to the data information;

第一发送模块，设置为将所述任务消息通过消息队列发送至所述业务模块，其中，所述任务消息用于指示所述业务模块对所述任务消息所对应的所述业务数据进行处理。The first sending module is configured to send the task message to the service module through a message queue, where the task message is used to instruct the service module to process the service data corresponding to the task message.
一种存储介质，所述存储介质包括存储的程序，其中，所述程序运行时执行上述权利要求1至7任一项中所述的方法。A storage medium including a stored program, wherein the method described in any one of claims 1 to 7 is executed when the program is running.
一种电子装置，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器通过所述计算机程序执行上述权利要求1至7任一项中所述的方法。An electronic device comprising a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the processor executes any one of claims 1 to 7 through the computer program The method described.