WO2019019400A1

WO2019019400A1 - Task distributed processing method, device, storage medium and server

Info

Publication number: WO2019019400A1
Application number: PCT/CN2017/104970
Authority: WO
Inventors: 严月强; 伍琼珂
Original assignee: 上海壹账通金融科技有限公司
Priority date: 2017-07-24
Filing date: 2017-09-30
Publication date: 2019-01-31
Also published as: CN107688496A; CN107688496B

Abstract

The invention relates to a task distributed processing method which is applied to a distributed task scheduling system. The distributed task scheduling system comprises a server and a plurality of terminals, wherein the server is a master node in the distributed task scheduling system, and each of the terminals is an execution node in the distributed task scheduling system. The method comprises: the server receives task processing requests transmitted by the terminals (S202); selecting at least one task in a to-be-distributed state, and distributing the selected tasks to the terminals for processing (S204); receiving task processing results fed back by the terminals; and modifying the task states of the corresponding tasks according to the task processing results (S206).

Description

任务分布式处理方法、装置、存储介质和服务器Task distributed processing method, device, storage medium and server

本申请要求于2017年07月24日提交中国专利局，申请号为201710608537X，发明名称为“任务分布式处理方法、装置、存储介质和服务器”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese Patent Application entitled "Task Distributed Processing Method, Apparatus, Storage Medium and Server", filed on July 24, 2017, with the application number of 201710608537X, the entire contents of which are incorporated by reference. Combined in this application.

技术领域Technical field

本申请涉及数据处理技术领域，特别是涉及一种任务分布式处理方法、装置、存储介质和服务器。The present application relates to the field of data processing technologies, and in particular, to a task distributed processing method, apparatus, storage medium, and server.

背景技术Background technique

分布式计算***通常都需要处理计算大量的任务，如需要进行统计日志报表、编译应用程序、解析Crash等。而处理这些分布式需要占用大量的服务器资源。Distributed computing systems often need to handle a large number of tasks, such as statistical log reporting, compiling applications, and parsing Crash. Handling these distributions requires a lot of server resources.

传统方法中，分布式计算***基本上都是架设在服务器集群上，基于Zookeeper将待处理的任务分配至服务器上的各个节点进行处理，使使得对任务的处理以及上分配进行优化，提高服务器的资源利用效率。然而服务器的资源终究是有限的，当需要处理的数据量较大、任务较多，甚至超过服务器的处理负荷时，传统的方法的任务处理效率并不高。In the traditional method, the distributed computing system is basically set up on the server cluster, and the tasks to be processed are allocated to the respective nodes on the server for processing according to Zookeeper, so that the processing of the task and the upper allocation are optimized, and the server is improved. Resource utilization efficiency. However, the resources of the server are limited. When the amount of data to be processed is large, the task is more, and even exceeds the processing load of the server, the task efficiency of the traditional method is not high.

发明内容Summary of the invention

根据本申请公开的各种实施例，提供一种任务分布式处理方法、装置、存储介质和服务器。According to various embodiments disclosed herein, a task distributed processing method, apparatus, storage medium, and server are provided.

一种任务分布式处理方法，应用于分布式任务调度***中，所述分布式任务调度***包括服务器和多个终端，所述服务器为所述分布式任务调度***中的主控节点，每个所述终端为所述分布式任务调度***中的执行节点；所述方法包括：所述服务器接收所述终端发送的任务处理请求；选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；接收所述终端反馈的任务处理结果；及根据所述任务处理结果修改对应任务的任务状态。A task distributed processing method is applied to a distributed task scheduling system, where the distributed task scheduling system includes a server and a plurality of terminals, wherein the server is a master node in the distributed task scheduling system, and each The terminal is an execution node in the distributed task scheduling system; The method includes: the server receiving a task processing request sent by the terminal; selecting at least one task to be assigned, assigning the selected task to the terminal for processing; and receiving a task processing result fed back by the terminal; Modifying the task status of the corresponding task according to the task processing result.

一种应用于分布式任务调度***中的任务分布式处理装置，包括：请求接收模块，用于接收终端发送的任务处理请求，所述终端为所述分布式任务调度***中的执行节点；任务分配模块，用于选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；及任务状态修改模块，用于接收所述终端反馈的任务处理结果；根据所述任务处理结果修改对应任务的任务状态。A task distributed processing device for use in a distributed task scheduling system, comprising: a request receiving module, configured to receive a task processing request sent by a terminal, where the terminal is an execution node in the distributed task scheduling system; a distribution module, configured to select at least one task to be assigned, to assign the selected task to the terminal for processing; and a task state modification module, configured to receive a task processing result fed back by the terminal; Modify the task status of the corresponding task.

一个或多个存储有计算机可读指令的计算机可读非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行一种任务分布式处理方法，所述方法应用于分布式任务调度***中，所述分布式任务调度***包括服务器和多个终端，所述服务器为所述分布式任务调度***中的主控节点，每个所述终端为所述分布式任务调度***中的执行节点；所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：接收所述终端发送的任务处理请求；选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；接收所述终端反馈的任务处理结果；及根据所述任务处理结果修改对应任务的任务状态。One or more computer readable non-volatile storage media storing computer readable instructions, when executed by one or more processors, causing the one or more processors to perform a task a distributed processing method, the method being applied to a distributed task scheduling system, the distributed task scheduling system comprising a server and a plurality of terminals, wherein the server is a master node in the distributed task scheduling system, each The terminal is an execution node in the distributed task scheduling system; when the computer readable instructions are executed by one or more processors, causing the one or more processors to perform the steps of: receiving the terminal to send The task processing request; selecting at least one task to be assigned, assigning the selected task to the terminal for processing; receiving the task processing result fed back by the terminal; and modifying the task status of the corresponding task according to the task processing result.

一种应用于分布式任务调度***中的服务器，包括存储器及处理器，所述存储器中储存有计算机可读指令，所述分布式任务调度***包括所述服务器和多个终端，所述服务器为所述分布式任务调度***中的主控节点，每个所述终端为所述分布式任务调度***中的执行节点；所述指令被所述处理器执行时，使得所述处理器执行以下步骤：接收所述终端发送的任务处理请求；选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；A server for use in a distributed task scheduling system includes a memory and a processor, wherein the memory stores computer readable instructions, the distributed task scheduling system includes the server and a plurality of terminals, and the server is a master node in the distributed task scheduling system, each of the terminals being an execution node in the distributed task scheduling system; when the instructions are executed by the processor, causing the processor to perform the following steps Receiving a task processing request sent by the terminal; selecting at least one task to be assigned, and assigning the selected task to the terminal for processing;

接收所述终端反馈的任务处理结果；及根据所述任务处理结果修改对应任务的任务状态。Receiving a task processing result fed back by the terminal; and modifying a task status of the corresponding task according to the task processing result.

本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。Details of one or more embodiments of the present application are set forth in the accompanying drawings and description below. This application Other features, objects, and advantages will be apparent from the description, drawings and claims.

附图说明DRAWINGS

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present application. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.

图1为一个实施例中任务分布式处理方法的应用环境图；1 is an application environment diagram of a task distributed processing method in an embodiment;

图2为一个实施例中任务分布式处理方法的流程图；2 is a flow chart of a task distributed processing method in an embodiment;

图3为一个实施例中对Crash解析任务的分布式处理方法的应用环境图FIG. 3 is an application environment diagram of a distributed processing method for Crash parsing tasks in an embodiment.

图4为另一个实施例中任务分布式处理方法的流程图；4 is a flow chart of a task distributed processing method in another embodiment;

图5为一个实施例中任务分布式处理装置的结构框图；FIG. 5 is a structural block diagram of a task distributed processing apparatus in an embodiment; FIG.

图6为另一个实施例中任务分布式处理装置的结构框图；及6 is a structural block diagram of a task distributed processing apparatus in another embodiment; and

图7为一个实施例中服务器的内部结构图。Figure 7 is a diagram showing the internal structure of a server in one embodiment.

具体实施方式Detailed ways

为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

本申请实施例所提供的任务分布式处理方法，可应用于如图1所示的应用环境中。参考图1，服务器110与多个终端120相连。其中，该终端可为笔记本电脑、台式电脑、平板等其中的任意一种。每个终端120均可向服务器发送任务处理请求。服务器110可根据所接收到任务处理请求，选取至少一个待分配状态的任务，将选取的任务分配给发送该任务处理请求的终端120进行处理。该终端120可对所接收到的任务进行分析处理，将任务处理结果发送给服务器110。服务器110根据接收到的任务处理结果修改对应任务的任务状态。从而实现了通过终端来对任务进行分布式处理。 The task distributed processing method provided by the embodiment of the present application can be applied to the application environment shown in FIG. 1 . Referring to FIG. 1, the server 110 is connected to a plurality of terminals 120. The terminal can be any one of a notebook computer, a desktop computer, a tablet, and the like. Each terminal 120 can send a task processing request to the server. The server 110 may select at least one task to be assigned according to the received task processing request, and allocate the selected task to the terminal 120 that sends the task processing request for processing. The terminal 120 may perform an analysis process on the received task, and send the task processing result to the server 110. The server 110 modifies the task status of the corresponding task according to the received task processing result. Thereby, distributed processing of tasks through the terminal is realized.

在一个实施例中，如图2所示，提供了一种任务分布式处理方法，该方法可应用于如图1所示的应用环境中，具体包括如下步骤：In an embodiment, as shown in FIG. 2, a task distributed processing method is provided, which can be applied to the application environment shown in FIG. 1, and specifically includes the following steps:

步骤S202，接收终端发送的任务处理请求。Step S202: Receive a task processing request sent by the terminal.

本实施例中，该终端为对应需要处理该任务的公司或集团内的员工终端或其它所属终端。每个终端可为DTS***(Distributed Task Scheduler，分布式任务调度***)中的一个任务执行节点。该DTS***为一个通用的、插件式的、支持扩展的分布式调度执行框架，用于实现对任务分布式分配和解析等处理。该任务是指数量众多的、每个任务可被分布式处理的任务，包括应用程序编译任务、日志报表统计任务、Crash解析任务等。In this embodiment, the terminal is an employee terminal or other belonging terminal in a company or a group corresponding to the task to be processed. Each terminal may be a task execution node in a DTS system (Distributed Task Scheduler). The DTS system is a general-purpose, plug-in, distributed extended execution framework that supports distributed processing and parsing of tasks. This task refers to a large number of tasks that each task can be distributed, including application compilation tasks, log report statistics tasks, Crash resolution tasks, and so on.

以Crash解析任务为例来说明，该Crash可为所收集的某一app(Application，应用)上报的Crash堆栈信息，可将1条或多条适应数量的待解析的Crash堆栈信息作为一个Crash解析任务。当收集Crash堆栈信息越多，则待处理的Crash解析任务也越多。Take the Crash parsing task as an example to illustrate that the Crash can report the Crash stack information reported by an app (application), and can analyze one or more adaptive Crash stack information to be parsed as a Crash. task. The more Crash stack information is collected, the more Crash parsing tasks are pending.

终端可在处于空闲状态时，可向该DTS***中的服务器发送任务处理请求，该任务处理请求中携带终端标识。该服务器为DTS***中的主控节点，负责管理任务和所有执行节点。任务的管理包含了接收、存储、分发、记录、监控任务；节点的管理包含了接收执行节点的心跳包、监控执行节点的工作状态等。服务器可实时接收每个终端所发送的任务处理请求，获取其中的终端标识，使得根据该终端标识确定对应的终端。When the terminal is in an idle state, the terminal may send a task processing request to the server in the DTS system, where the task processing request carries the terminal identifier. The server is the master node in the DTS system and is responsible for managing tasks and all execution nodes. The management of the task includes receiving, storing, distributing, recording, and monitoring tasks; the management of the node includes receiving the heartbeat packet of the executing node, monitoring the working state of the executing node, and the like. The server can receive the task processing request sent by each terminal in real time, and obtain the terminal identifier in the terminal, so that the corresponding terminal is determined according to the terminal identifier.

步骤S204，选取至少一个待分配状态的任务，将选取的任务分配给终端进行处理。Step S204: Select at least one task to be assigned, and assign the selected task to the terminal for processing.

本实施例中，服务器可从数据库所存储的待分配状态的任务中，选取预设数量的任务，将选取的任务的任务信息发送至终端，使得终端对该任务进行处理。其中，任务的状态包括待分配、处理中、处理成功以及处理失败的状态。预设数量可为预设的1个或几个等合适的数量，以使得终端可同时对多个任务进行处理，提高处理的效率。对待分配状态的任务的选取方式可包括随机选取方式以及按照任务生成的时间顺序来进行选取的方式。 In this embodiment, the server may select a preset number of tasks from the tasks to be allocated in the database, and send the task information of the selected task to the terminal, so that the terminal processes the task. The status of the task includes the status to be assigned, in process, successful in processing, and failed in processing. The preset number may be a preset number or a plurality of suitable numbers, so that the terminal can process multiple tasks at the same time, thereby improving the processing efficiency. The manner of selecting the task to be assigned may include a random selection method and a manner of selecting according to the time sequence generated by the task.

具体地，终端可接收所分配的任务，并按照对应的任务解析方式对该任务进行解析处理，将处理结果发送给服务器。Specifically, the terminal may receive the allocated task, parse the task according to the corresponding task resolution manner, and send the processing result to the server.

在一个实施例中，在步骤S204之后，还包括：将选取的任务的状态修改为处理中。In an embodiment, after step S204, the method further comprises: modifying the state of the selected task to be processed.

步骤S206，接收终端反馈的任务处理结果；根据任务处理结果修改对应任务的任务状态。Step S206: Receive a task processing result fed back by the terminal, and modify a task state of the corresponding task according to the task processing result.

本实施例中，服务器可接收终端对该任务进行处理后，所反馈的任务处理结果。该任务处理结果包括任务处理成功和任务处理失败。服务器可根据该任务处理结果修改对应已分配的任务的任务状态。比如，当该任务为处理成功，则将其任务状态修改为处理成功，若为处理失败，则将其任务状态修改为处理失败。In this embodiment, the server may receive the task processing result that is fed back after the terminal processes the task. The task processing results include task processing success and task processing failure. The server may modify the task status of the corresponding assigned task according to the task processing result. For example, when the task is successfully processed, its task status is modified to be processed successfully. If the processing fails, its task status is changed to processing failure.

在一个实施例中，每个终端之间可为在任务处理上相互独立的终端。即在任务处理的过程中，终端之间进行互相独立的处理，并无连接关系。在DTS***中，由于每个终端仅与服务器相连，可保持对任务处理的独立性，以提高终端进行任务处理的效率。In one embodiment, each terminal may be a terminal that is independent of each other in task processing. That is, in the process of task processing, the terminals are processed independently of each other, and there is no connection relationship. In the DTS system, since each terminal is only connected to the server, the independence of the task processing can be maintained to improve the efficiency of the terminal for task processing.

本实施例中，通过接收终端发送的任务处理请求；选取至少一个待分配状态的任务，将选取的任务分配给终端进行处理；然后接收终端反馈的任务处理结果；根据任务处理结果修改对应任务的任务状态。从而实现了将终端作为DTS***中的任务执行节点，对任务进行处理。当终端数量越多，则服务器上可被及时分配的分布式处理的任务也越多。从而使得无需增加服务器，或者对服务器进行扩容，通过利用已有的终端，将众多的任务分配给每个已有的终端进行处理，提高了任务处理的效率。In this embodiment, the task processing request sent by the terminal is received; at least one task to be assigned is selected, and the selected task is allocated to the terminal for processing; then the task processing result fed back by the terminal is received; and the corresponding task is modified according to the task processing result. Task status. Thereby, the terminal is implemented as a task execution node in the DTS system, and the task is processed. The greater the number of terminals, the more distributed processing tasks that can be allocated on the server in a timely manner. Therefore, the server is not required to be added, or the server is expanded, and a plurality of tasks are allocated to each existing terminal for processing by using the existing terminal, thereby improving the efficiency of task processing.

在一个实施例中，上述的选取至少一个待分配状态的任务，包括：选取至少一个待分配状态的任务标识；从主数据库中读取每个任务标识对应的任务，当从主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。In an embodiment, the selecting the at least one task to be assigned includes: selecting at least one task identifier to be assigned; reading the task corresponding to each task identifier from the primary database, when reading from the primary database fails When the task corresponding to each task ID is read from the standby database.

本实施例中，任务标识用于唯一识别对应的任务，并与对应任务的任务信息相关联，可由预设位数的数字、字母等字符所构成。任务标识可为任务编号。服务器可在检测到有任务生成时，可为所生成的任务设置相应的任务编号，并将该任务信息分别写入主数据库和备用数据库中。In this embodiment, the task identifier is used to uniquely identify the corresponding task, and the task corresponding to the task The information is associated with a number of characters, letters, and the like of the preset number of digits. The task ID can be the task number. When the server detects that a task is generated, the server can set a corresponding task number for the generated task, and write the task information into the primary database and the standby database, respectively.

进一步地，还预先设置了两个数据库，分别为主数据库和备用数据库。每个未处理完成的任务的任务信息在主数据库和备用数据库中均有存储，并可根据该任务编号从主数据库或备用数据库中查询出来。主数据库为默认优先使用的数据库，备用数据库为备用的数据库。服务器可在选取了任务标识后，可根据该任务标识首先从主数据库中查询对应的任务，并读取该任务信息。当从主数据库中读取失败时，则从备用数据库中读取对应任务的任务信息，从而任可进一步提高对任务的提取的成功率。Further, two databases are set in advance, which are a primary database and a standby database, respectively. The task information of each unprocessed task is stored in both the primary database and the standby database, and can be queried from the primary database or the standby database according to the task number. The primary database is the default priority database and the standby database is the standby database. After selecting the task identifier, the server may first query the corresponding task from the primary database according to the task identifier, and read the task information. When the reading from the primary database fails, the task information of the corresponding task is read from the standby database, so that the success rate of the extraction of the task can be further improved.

本实施例中，通过提供主数据库和备用数据库，从而可在主数据库出现故障时，从备用数据库中读取待分配的任务。可防止数据库出现故障，导致待处理的任务积压而造成进一步的影响。In this embodiment, by providing the primary database and the standby database, the task to be allocated can be read from the standby database when the primary database fails. It can prevent the database from malfunctioning, resulting in a backlog of pending tasks and further impact.

在一个实施例中，任务处理请求中携带终端的互联网协议地址；将选取的任务分配给终端进行处理，包括：根据互联网协议地址，将选取的任务分配给终端进行处理。In an embodiment, the task processing request carries the Internet Protocol address of the terminal; and the selected task is allocated to the terminal for processing, including: assigning the selected task to the terminal for processing according to the Internet Protocol address.

本实施例中，互联网协议地址(Internet Protocol Address，IP地址)为终端的本机IP地址。由于终端为员工终端，需要处理其它工作事项，可能存在随时关机、重启、断网、切换网络等操作，使得终端的IP地址或其它参数特征出现变化。相比较于传统的zookeeper需要在启动之前事先配置好所有执行节点的IP和端口。本实施例直接通过在任务处理请求中携带终端的IP地址，使得服务器可向该IP地址发送所分配的任务，实现了即时终端的IP地址出现了动态变化，也可根据变化后的IP地址，成功将任务信息发送至终端，使终端进行任务处理。In this embodiment, the Internet Protocol Address (IP address) is the local IP address of the terminal. Because the terminal is an employee terminal and needs to handle other work items, there may be operations such as shutting down, restarting, disconnecting the network, and switching the network at any time, so that the IP address or other parameter characteristics of the terminal change. Compared to the traditional zookeeper, you need to configure the IP and port of all the execution nodes before starting. In this embodiment, the IP address of the terminal is carried in the task processing request, so that the server can send the assigned task to the IP address, and the IP address of the instant terminal is dynamically changed, and the changed IP address can also be used according to the changed IP address. The task information is successfully sent to the terminal, and the terminal performs task processing.

在一个实施例中，在将选取的任务分配给终端进行处理之后，包括：接收终端发送的心跳包，以根据心跳包确定终端正在处理选取的任务。In an embodiment, after the selected task is allocated to the terminal for processing, the method includes: receiving a heartbeat packet sent by the terminal, to determine, according to the heartbeat packet, that the terminal is processing the selected task.

本实施例中，终端可在接收到所分配的任务后，按照预设的频率向服务器发送心跳包，该心跳包中包含终端的IP地址，使得服务器可根据所接收到的每个心跳包，识别出对应的IP地址，从而可确定对应的终端。服务器通过接收每个终端定时发送的心跳包，可获知对应终端正在处理被分配的任务，从而还实现了对每个终端进行任务处理的监控。In this embodiment, the terminal may serve the service according to a preset frequency after receiving the assigned task. The heartbeat packet is sent, and the heartbeat packet includes the IP address of the terminal, so that the server can identify the corresponding IP address according to each heartbeat packet received, so that the corresponding terminal can be determined. By receiving the heartbeat packet periodically sent by each terminal, the server can know that the corresponding terminal is processing the assigned task, thereby implementing monitoring of task processing for each terminal.

在一个实施例中，在将选取的任务分配给终端进行处理之后，包括：检测是否在预设时长之内接收到任务处理结果，若否，则将选取的任务重新分配给其它发送任务处理请求的终端进行处理。In an embodiment, after the selected task is allocated to the terminal for processing, the method includes: detecting whether the task processing result is received within a preset duration, and if not, reassigning the selected task to another sending task processing request. The terminal is processed.

本实施例中，服务器还设置了时长阈值，即上述的预设时长。该时长阈值为用于评判任务的处理是否异常的时长。时长阈值可根据历史统计出的完成任务的处理所需的平均时长，而设置的时长。可为超过该平均时长一定大小的时长。比如处理一个任务的平均时长为5分钟，则可设置该时长阈值可为8分钟或10分钟等。In this embodiment, the server further sets a duration threshold, that is, the preset duration described above. The duration threshold is a duration for judging whether the processing of the task is abnormal. The duration threshold can be set according to the historical average time required to complete the processing of the task. It may be a length of time that exceeds the average length. For example, if the average duration of processing a task is 5 minutes, the duration threshold can be set to 8 minutes or 10 minutes.

服务器可在分配了任务给终端后，开始实时统计任务的处理时长，并比较该处理时长和预设时长的大小，当该处理时长达到预设时长，且还未接收到终端发送的任务处理结果时，则判定终端对该任务的处理失败。并重新将所分配任务的任务状态设置为待分配状态，并根据所接收到的其它终端的任务处理请求，重新对该任务进行分配。After the task is assigned to the terminal, the server can start the processing time of the real-time statistical task, and compare the processing duration and the preset duration. When the processing duration reaches the preset duration, the server does not receive the task processing result sent by the terminal. Then, it is determined that the processing of the task by the terminal fails. And re-set the task status of the assigned task to the status to be assigned, and re-allocate the task according to the received task processing request of other terminals.

本实施例中，通过设置预设时长，将超过预设时长还未收到任务处理结果的任务进行重新分配，可进一步提到对任务处理完成的成功率。In this embodiment, by setting a preset duration, and re-allocating a task that has not received the task processing result by the preset duration, the success rate of the task processing completion may be further mentioned.

在一个实施例中，上述的任务分布式处理方法还包括：计算选取的任务的分配次数；当分配次数达到预设次数时，若在预设时长之内未接收到任务处理结果，或者接收到任务处理失败的任务处理结果，则将选取任务的任务状态修改为处理异常的状态。In an embodiment, the task distributed processing method further includes: calculating a number of allocations of the selected tasks; and when the number of allocations reaches a preset number of times, if the task processing result is not received within the preset duration, or is received If the task processing fails the task processing result, the task status of the selected task is modified to handle the abnormal state.

本实施例中，当产生任务被重新分配时，服务器可进一步统计被重新分配的任务的分配次数，并比较该分配次数和预设次数的大小。其中，预设次数可为设置的合适的次数，比如可为3次或5次等。In this embodiment, when the generation task is reassigned, the server may further count the number of allocations of the reassigned tasks, and compare the number of allocations with the preset number of times. The preset number of times may be a suitable number of times, for example, may be 3 times or 5 times.

当出现被分配次数达到预设次数的任务时，还未接收到任务处理成功的任务处理结果，产生的原因可能为任务本身出现问题，难以被成功处理。因此，可判定该任务被多次处理失败，则可终止对该任务的重新分配，将该任务的状态设置为处理异常的状态。When a task that has been assigned a preset number of times occurs, the task has not been successfully processed. The result of the task processing may be caused by problems in the task itself and difficult to be successfully processed. Therefore, if it can be determined that the task has failed to be processed multiple times, the reallocation of the task can be terminated, and the state of the task is set to a state in which the abnormality is handled.

在一个实施例中，服务器可汇总每个被设置为处理异常的状态的任务信息，将该任务信息发送至对应的管理员终端，使相应的管理员对该任务进行进一步地检测。具体地，可按照预设的汇总频率进行汇总，比如按照每小时一次的频率进行汇总。In one embodiment, the server may summarize task information that is set to handle the abnormal state, send the task information to the corresponding administrator terminal, and cause the corresponding administrator to further detect the task. Specifically, the aggregation may be performed according to a preset summary frequency, for example, by frequency once every hour.

本实施例中，通过设置预设次数，可防止任务被无限地重新分配处理，以浪费终端和服务器的资源。In this embodiment, by setting a preset number of times, the task can be prevented from being redistributed indefinitely, thereby wasting resources of the terminal and the server.

在一个实施例中，上述的方法还包括：统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。In an embodiment, the foregoing method further includes: counting the number of tasks completed by each terminal, and sorting the terminals according to the number of completed tasks.

服务器可统计在预设时间段内，每个终端完成的任务数量。具体地，可根据所接收到的任务处理请求中携带的终端标识，统计具有相同终端标识的终端所反馈的任务处理成功的任务处理结果的数量。其中，该终端标识可为用户名等可唯一识别终端身份的信息。The server can count the number of tasks completed by each terminal within a preset time period. Specifically, the number of task processing results successfully processed by the terminal with the same terminal identifier may be counted according to the terminal identifier carried in the received task processing request. The terminal identifier may be information such as a user name that can uniquely identify the identity of the terminal.

比如可按周或者天为单位，统计每天或每周之内，每个终端完成的任务数量，并根据完成的任务数量进行排序。其中，排序信息中包括终端标识、完成的任务数量以及排名等信息。For example, the number of tasks completed by each terminal can be counted on a per-weekly or daily basis, and sorted according to the number of completed tasks. The sorting information includes information such as a terminal identifier, a number of completed tasks, and a ranking.

本实施例中，通过对每个终端完成的任务数量进行排序，使得每个终端之间形成一种竞赛关系，具有互动性，可提高终端用户参与任务处理的积极性呵和数量，以进一步提高了对任务处理效率。In this embodiment, by sorting the number of tasks completed by each terminal, a competition relationship is formed between each terminal, which is interactive, and the enthusiasm and quantity of the terminal users participating in the task processing can be improved, thereby further improving. The efficiency of task processing.

进一步地，可将排序信息发送至对应的终端，使得终端可获知其在预设时间段内完成的任务数量以及排名。更进一步地，服务器还可从排序中选取中排序超过预设名次的、或者完成的任务数量超过预设数量的终端，并向每个终端广播对所选取的终端的奖励信息，以进一步提高每个终端用户参与任务处理的积极性。 Further, the ranking information may be sent to the corresponding terminal, so that the terminal can know the number of tasks and rankings that are completed in the preset time period. Further, the server may further select, from the sorting, the number of tasks that are more than the preset number, or the number of completed tasks exceeds the preset number, and broadcast the reward information to the selected terminal to each terminal to further improve each The enthusiasm of end users to participate in task processing.

在一个实施例中，上述的任务为Crash解析任务。如图3所示，为一个实施例中对Crash解析任务的分布式处理方法的应用环境图，包括服务器110，终端120，数据库130以及Crash解析平台。其中，服务器110中包含主服务器112以及备用服务器112，数据库同样包含主数据库132以及备用数据库134。Crash解析平台为Crash数据提供者，Crash解析任务需要获取和上传Crash文件，该Crash解析平台负责提供Crash源数据并接收解析后的结果上传。每个终端120均可向服务器110发送任务处理请求，服务器110根据所接收到的任务处理请求，从数据库130中选取至少一个待分配状态的任务，通过与该终端110所定义的接口，分配给该终端110，该接口可为http实现的接口。终端110接收服务器110发送的任务信息，并与Crash平台进行交互，以实现对Crash的解析处理。终端110将任务处理结果发送至服务器110，服务器110根据任务处理结果修改对应任务的任务状态，从而实现了对Crash解析任务的处理。In one embodiment, the above task is a Crash parsing task. As shown in FIG. 3, an application environment diagram of a distributed processing method for a Crash parsing task in an embodiment includes a server 110, a terminal 120, a database 130, and a Crash parsing platform. The server 110 includes a primary server 112 and a backup server 112. The database also includes a primary database 132 and a standby database 134. The Crash parsing platform is a Crash data provider. The Crash parsing task needs to obtain and upload a Crash file. The Crash parsing platform is responsible for providing the Crash source data and receiving the parsed result upload. Each terminal 120 can send a task processing request to the server 110, and the server 110 selects at least one task to be assigned from the database 130 according to the received task processing request, and allocates to the interface defined by the terminal 110 to the task. The terminal 110 can be an interface implemented by http. The terminal 110 receives the task information sent by the server 110 and interacts with the Crash platform to implement parsing processing of the Crash. The terminal 110 sends the task processing result to the server 110, and the server 110 modifies the task status of the corresponding task according to the task processing result, thereby implementing the processing of the Crash parsing task.

在一个实施例中，如图4所示，提供了另一种任务分布式处理方法，该任务为Crash解析任务。该方法包括：In one embodiment, as shown in FIG. 4, another task distributed processing method is provided, which is a Crash parsing task. The method includes:

步骤S401，终端向服务器发送任务处理请求，任务处理请求中携带终端的互联网协议地址。Step S401: The terminal sends a task processing request to the server, where the task processing request carries the Internet Protocol address of the terminal.

本实施例中，该任务处理请求为Crash解析任务的任务处理请求。进一步地，可为IOS Crash解析任务的任务处理请求，终端为部署了Mac OSX***的终端，比如为Mac笔记本或Mac台式电脑等。In this embodiment, the task processing request is a task processing request of the Crash parsing task. Further, the task can be processed for the IOS Crash resolution task, and the terminal is a terminal that deploys a Mac OSX system, such as a Mac notebook or a Mac desktop computer.

服务器可预先或实时生成了多个Crash解析任务，并将该写入主数据库以及备用数据库中。同时对所生成的Crash解析任务设置任务标识以及设置任务状态为待分配状态。其中，Crash解析任务中包含待解析的Crash文件。具体地，可提供接口和web页面创建任务，从而可接收终端通过该接口和web页面创建的Crash解析任务。The server can generate multiple Crash resolution tasks in advance or in real time and write this into the primary database as well as the standby database. At the same time, the task identifier is set for the generated Crash parsing task and the task status is set to be assigned. Among them, the Crash parsing task contains the Crash file to be parsed. Specifically, an interface and a web page creation task may be provided, so that the Crash resolution task created by the terminal through the interface and the web page may be received.

在一个实施例中，服务器可将多个Crash文件封装成一个压缩包，并将多个压缩包封装成一个Crash解析任务，从而使得分配给终端的一个Crash 解析任务中包含多个待解析的Crash文件，从而可提高了Crash文件的分配处理效率。In one embodiment, the server may encapsulate multiple Crash files into one compressed package and encapsulate multiple compressed packages into one Crash parsing task, thereby causing a Crash to be assigned to the terminal. The parsing task contains multiple Crash files to be parsed, which can improve the allocation processing efficiency of Crash files.

在一个实施例中，终端可检测自身是否处于空闲状态，若是，则向服务器发送任务处理请求。其中空闲状态是指包括CPU利用率低于预设利用率、内存占用率低于预设占用率的状态。其中，预设利用率和预设占用率可设置为任意合适的数值，比如可均为50％。通过在处于空闲状态时，才发送任务处理请求，可降低处理任务对终端的正常工作造成影响。In one embodiment, the terminal can detect if it is in an idle state, and if so, send a task processing request to the server. The idle state includes a state in which the CPU utilization is lower than the preset utilization rate and the memory usage is lower than the preset occupancy rate. The preset utilization rate and the preset occupancy rate may be set to any suitable value, for example, may be 50%. By sending a task processing request when it is in an idle state, the processing task can be reduced to affect the normal operation of the terminal.

步骤S402，服务器接收该任务处理请求，选取至少一个待分配状态的任务，根据互联网协议地址，将选取的任务分配给终端进行处理。Step S402, the server receives the task processing request, selects at least one task to be assigned, and assigns the selected task to the terminal for processing according to the Internet Protocol address.

具体地，服务器可任务生成的时间顺序来进行选取预设数量的待分配的任务的任务标识，并优先从主数据库中查询该任务标识所对应的任务。读取所查询出的任务，并按照任务处理请求中携带的终端的IP地址，将其发送至对应的终端，使终端进行任务处理。在一个实施例中，服务器可预设用于存放待分配任务的第一任务表，该表中可按任务的生成顺序来存储待分配任务的任务信息。具体地，可存储任务的任务标识。根据终端的任务处理请求，按照生成顺序从该表中选取还未分配的任务的任务标识。根据该任务标识从主数据库中读取对应的任务信息。其中，当从主数据库中读取失败时，可从备用数据库中读取该任务。Specifically, the server may select a preset number of task identifiers of the tasks to be assigned, and query the tasks corresponding to the task identifiers from the primary database in priority. The queried task is read, and the IP address of the terminal carried in the request processing request is sent to the corresponding terminal, so that the terminal performs task processing. In an embodiment, the server may preset a first task table for storing tasks to be assigned, where the task information of the tasks to be assigned may be stored in the order in which the tasks are generated. Specifically, the task identifier of the task can be stored. According to the task processing request of the terminal, the task identifier of the task that has not been assigned is selected from the table in the order of generation. The corresponding task information is read from the primary database according to the task identifier. Wherein, when the read from the primary database fails, the task can be read from the standby database.

在一个实施例中，该服务器可为一个服务器集群，包含主服务器和备用服务器。可默认调用主服务器进行任务分配和监控等处理，当检测到主服务器处理出现故障时，可调用备用服务器进行任务分配和监控等处理。In one embodiment, the server can be a server cluster containing a primary server and a secondary server. The main server can be called by default for task allocation and monitoring. When it is detected that the main server is faulty, the standby server can be called to perform task allocation and monitoring.

在一个实施例中，在将选取的任务分配给终端进行处理之后，还包括：将已分配的任务的状态修改为处理中。进一步地，可分配任务的数据库逻辑设置事务隔离，从而避免将同一个任务分配给了多个终端而重复执行的情况发生。In an embodiment, after the selected task is allocated to the terminal for processing, the method further includes: modifying the state of the assigned task to be processed. Further, the database logic of the assignable task sets transaction isolation, thereby avoiding the case where the same task is assigned to multiple terminals and repeated execution occurs.

在一个实施例中，可将该任务从第一任务表中移动到用于存放处理中的第二任务表中，或者直接在第一任务表中将该任务的任务状态标记为处理中的任务状态。In one embodiment, the task may be moved from the first task table to the second task table for storing the process, or the task status of the task may be marked as being processed directly in the first task table. Task status.

步骤S403，终端接收服务器分配的任务，通过Crash解析平台对该任务进行处理，将任务处理结果发送至服务器。Step S403, the terminal receives the task assigned by the server, processes the task through the Crash analysis platform, and sends the task processing result to the server.

本实施例中，终端在接收到服务器分配的任务后，可通过与Crash进行交互，从而实现对任务中的每个Crash文件的解析处理，生成任务处理结果并发送至服务器。其中，该任务处理结果中可包含每个Crash文件的处理结果。处理结果包括处理成功和处理失败。In this embodiment, after receiving the task assigned by the server, the terminal can interact with the Crash to implement parsing processing of each Crash file in the task, generate a task processing result, and send the result to the server. The task processing result may include the processing result of each Crash file. Processing results include processing success and processing failure.

在一个实施例中，终端还可按照预设的频率向服务器发送心跳包，以告知终端正在进行任务的解析处理。In an embodiment, the terminal may also send a heartbeat packet to the server according to a preset frequency to inform the terminal that the parsing process of the task is in progress.

步骤S404，服务器检测是否在预设时长之内接收到任务处理结果，若是，则执行步骤S405，否则，执行步骤S406。In step S404, the server detects whether the task processing result is received within the preset duration. If yes, step S405 is performed; otherwise, step S406 is performed.

本实施例中，服务器可在将选取的任务分配给终端进行处理之后，则开始统计任务的处理时长，若在预设时长之内接收到任务处理结果。任务处理结果中包含对应的任务标识，使得服务器可根据该任务标识任务处理结果所属的任务。In this embodiment, after the server assigns the selected task to the terminal for processing, the server starts to process the processing time of the task, and if the task processing result is received within the preset duration. The task processing result includes a corresponding task identifier, so that the server can identify the task to which the task processing result belongs according to the task.

服务器可提取将所有状态为处理中，且分配时间距离当前时间已经超出预设时长的任务，然后重置该任务的状态为待分配。预设时长可根据经验值设定，假设一个任务正常执行时间为2分钟，最长执行时间为5分钟，则可设置预设时长为6分钟或者8分钟。The server may extract a task that has all the states as being processed, and the allocated time is longer than the preset time, and then resets the state of the task to be allocated. The preset duration can be set according to the experience value. Assuming that the normal execution time of a task is 2 minutes and the maximum execution time is 5 minutes, the preset duration can be set to 6 minutes or 8 minutes.

步骤S405，根据任务处理结果修改对应任务的任务状态。Step S405, modifying the task status of the corresponding task according to the task processing result.

步骤S406，将选取的任务重新分配给其它发送任务处理请求的终端进行处理。In step S406, the selected task is reassigned to other terminals that send the task processing request for processing.

具体地，若任务状态为处理失败，则将该任务再修改为待分配的状态，并再次存放入第一任务表中，以进行重新分配。若任务状态为处理成功，则将该任务存放入用于存储处理成功的第三任务表中。若任务状态为处理异常，则将该任务存放如用于存储处理异常的第四任务表中。其中，每个任务表中均包含对应任务的任务标识，以便于服务器可定时根据第四任务表记录的任务标识，从主数据库或备用数据库中读取对应的任务信息，发送至管理员终端，使管理员进行检测。Specifically, if the task status is a processing failure, the task is further modified into a state to be allocated, and is again stored in the first task table for reallocation. If the task status is successful, the task is placed in a third task table for storing the processing success. If the task status is a processing exception, the task is stored in a fourth task table for storing processing exceptions. Each task table includes a task identifier of the corresponding task, so that the server can periodically record according to the fourth task table. The service identifier reads the corresponding task information from the primary database or the standby database and sends it to the administrator terminal for the administrator to detect.

在一个实施例中，终端可设置多个线程来进行任务分配请求与处理。比如可设置2个线程，每个线程均在循环地“任务请求-＞执行-＞处理结果上报-＞休息预设时长”，从而提高终端的利用率和任务处理效率。In one embodiment, the terminal may set up multiple threads for task assignment requests and processing. For example, two threads can be set, and each thread is cyclically "task request -> execution -> processing result report -> rest preset duration", thereby improving terminal utilization and task processing efficiency.

上述的任务分布式处理方法，通过将终端作为DTS***中的执行节点，将任务分配给多个终端进行处理，可缓解服务器对任务处理的压力，提高了对任务处理的效率。The above-mentioned task distributed processing method can allocate the task to multiple terminals for processing by using the terminal as an execution node in the DTS system, which can alleviate the pressure of the server on the task processing and improve the efficiency of the task processing.

在一个实施例中，如图5所示，提供了一种任务分布式处理装置，该装置应用于分布式任务调度***中的，包括：In one embodiment, as shown in FIG. 5, a task distributed processing apparatus is provided, which is applied to a distributed task scheduling system, and includes:

请求接收模块502，用于接收终端发送的任务处理请求，所述终端为所述分布式任务调度***中的执行节点。The request receiving module 502 is configured to receive a task processing request sent by the terminal, where the terminal is an execution node in the distributed task scheduling system.

任务分配模块504，用于选取至少一个待分配状态的任务，将选取的任务分配给终端进行处理。The task assignment module 504 is configured to select at least one task to be assigned, and assign the selected task to the terminal for processing.

任务状态修改模块506，用于接收终端反馈的任务处理结果；根据任务处理结果修改对应任务的任务状态。The task status modification module 506 is configured to receive a task processing result fed back by the terminal, and modify a task status of the corresponding task according to the task processing result.

在一个实施例中，任务分配模块504还用于选取至少一个待分配状态的任务标识；从主数据库中读取每个任务标识对应的任务，当从主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。In an embodiment, the task assignment module 504 is further configured to select at least one task identifier to be assigned status; read a task corresponding to each task identifier from the primary database, and when the read from the primary database fails, from the standby database. Read the task corresponding to each task ID.

在一个实施例中，任务处理请求中携带终端的互联网协议地址；任务分配模块504还用于根据互联网协议地址，将选取的任务分配给终端进行处理。In one embodiment, the task processing request carries the Internet Protocol address of the terminal; the task assignment module 504 is further configured to allocate the selected task to the terminal for processing according to the Internet Protocol address.

在一个实施例中，任务状态修改模块506还用于接收终端发送的心跳包，以根据心跳包确定终端正在处理选取的任务。In one embodiment, the task status modification module 506 is further configured to receive a heartbeat packet sent by the terminal to determine, according to the heartbeat packet, that the terminal is processing the selected task.

在一个实施例中，任务分配模块504还用于检测是否在预设时长之内接收到任务处理结果，若否，则将选取的任务重新分配给其它发送任务处理请求的终端进行处理。 In one embodiment, the task assignment module 504 is further configured to detect whether the task processing result is received within a preset duration, and if not, re-allocate the selected task to another terminal that sends the task processing request for processing.

在一个实施例中，如图6所示，该装置还包括：In one embodiment, as shown in FIG. 6, the apparatus further includes:

排序模块508，用于统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。The sorting module 508 is configured to count the number of tasks completed by each terminal, and sort the terminals according to the number of completed tasks.

上述任务分布式处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。其中，网络接口可以是以太网卡或无线网卡等。上述各模块可以硬件形式内嵌于或独立于服务器中的处理器中，也可以以软件形式存储于服务器中的存储器中，以便于处理器调用执行以上各个模块对应的操作。该处理器可以为中央处理单元(CPU)、微处理器、单片机等。The various modules in the above-described task distributed processing device may be implemented in whole or in part by software, hardware, and combinations thereof. The network interface may be an Ethernet card or a wireless network card. The above modules may be embedded in the hardware in the processor or in the memory in the server, or may be stored in the memory in the server, so that the processor calls the corresponding operations of the above modules. The processor can be a central processing unit (CPU), a microprocessor, a microcontroller, or the like.

在一个实施例中，本申请提供的任务分布式处理装置可以实现为一种计算机程序的形式，该计算机程序可在如图7所示的服务器上运行，该服务器的非易失性存储介质可存储组成该任务分布式处理装置的各个程序模块。比如可包括如图5所示的请求接收模块502、任务分配模块504及任务状态修改模块506。各个程序模块中包括计算机可读指令，该计算机可读指令用于使该服务器执行本说明书中描述的本申请各个实施例的任务分布式处理方法中的步骤，例如，服务器可以通过如图5所示的任务分布式处理装置中的行为请求接收模块502接收终端发送的任务处理请求，所述终端为所述分布式任务调度***中的执行节点；通过任务分配模块选取至少一个待分配状态的任务，将选取的任务分配给终端进行处理；及通过任务状态修改模块506接收终端反馈的任务处理结果；根据任务处理结果修改对应任务的任务状态。In one embodiment, the task distributed processing apparatus provided by the present application may be implemented in the form of a computer program, which may be run on a server as shown in FIG. 7, and the non-volatile storage medium of the server may be The various program modules that make up the task distributed processing device are stored. For example, the request receiving module 502, the task assigning module 504, and the task state modifying module 506 as shown in FIG. 5 may be included. Each of the program modules includes computer readable instructions for causing the server to perform the steps in the task distributed processing method of various embodiments of the present application described in the present specification, for example, the server may be as shown in FIG. The behavior request receiving module 502 in the illustrated task distributed processing device receives the task processing request sent by the terminal, the terminal is an execution node in the distributed task scheduling system; and at least one task to be assigned is selected by the task assignment module The task is allocated to the terminal for processing; and the task state modification module 506 receives the task processing result fed back by the terminal; and the task state of the corresponding task is modified according to the task processing result.

在一个实施例中，提供了一个或多个包含计算机可读指令的非易失性计算机可读存储介质，当该计算机可读指令被一个或多个处理器执行时，使得该处理器实现一种任务分布式处理方法，该方法应用于分布式任务调度***中，该分布式任务调度***包括服务器和多个终端，该服务器为所述分布式任务调度***中的主控节点，每个终端为该分布式任务调度***中的执行节点，该处理器执行所述指令时执行上述各个实施例所提供的任务分布式处理方法的步骤。 In one embodiment, one or more non-transitory computer readable storage media containing computer readable instructions are provided that, when executed by one or more processors, cause the processor to implement a A task distributed processing method, the method is applied to a distributed task scheduling system, the distributed task scheduling system comprising a server and a plurality of terminals, wherein the server is a master node in the distributed task scheduling system, and each terminal For the execution node in the distributed task scheduling system, the processor executes the steps of the task distributed processing method provided by the various embodiments described above when the instruction is executed.

在一个实施例中，当该计算机可读指令被一个或多个处理器执行时，使得该处理器执行以下步骤：接收终端发送的任务处理请求；选取至少一个待分配状态的任务，将选取的任务分配给终端进行处理；接收终端反馈的任务处理结果；根据任务处理结果修改对应任务的任务状态。In one embodiment, when the computer readable instructions are executed by one or more processors, the processor is caused to perform the steps of: receiving a task processing request sent by the terminal; selecting at least one task to be assigned, selecting the selected The task is allocated to the terminal for processing; the task processing result fed back by the terminal is received; and the task state of the corresponding task is modified according to the task processing result.

在一个实施例中，当该计算机可读指令被一个或多个处理器执行时，使得该处理器执行的选取至少一个待分配状态的任务，包括：选取至少一个待分配状态的任务标识；从主数据库中读取每个任务标识对应的任务，当从主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。In one embodiment, when the computer readable instructions are executed by one or more processors, causing the processor to perform at least one task to be assigned a state, comprising: selecting at least one task identifier to be assigned a status; The task corresponding to each task identifier is read in the primary database. When the read from the primary database fails, the task corresponding to each task identifier is read from the standby database.

在一个实施例中，任务处理请求中携带终端的互联网协议地址；当该计算机可读指令被一个或多个处理器执行时，使得该处理器执行的将选取的任务分配给终端进行处理，包括：根据互联网协议地址，将选取的任务分配给终端进行处理。In one embodiment, the task processing request carries an internet protocol address of the terminal; when the computer readable instruction is executed by the one or more processors, causing the processor to perform the task of assigning the selected task to the terminal for processing, including : Assign the selected task to the terminal for processing according to the Internet Protocol address.

在一个实施例中，当该计算机可读指令被一个或多个处理器执行时，使得该处理器在执行将选取的任务分配给终端进行处理之后，还包括执行以下步骤：接收终端发送的心跳包，以根据心跳包确定终端正在处理选取的任务。In one embodiment, when the computer readable instructions are executed by one or more processors, causing the processor to perform the process of assigning the selected task to the terminal for processing, further comprising the step of: receiving a heartbeat sent by the terminal Packet to determine that the terminal is processing the selected task based on the heartbeat packet.

在一个实施例中，当该计算机可读指令被一个或多个处理器执行时，使得该处理器在执行将选取的任务分配给终端进行处理之后，还包括执行以下步骤：检测是否在预设时长之内接收到任务处理结果，若否，则将选取的任务重新分配给其它发送任务处理请求的终端进行处理。In one embodiment, when the computer readable instructions are executed by one or more processors, causing the processor to perform the process of assigning the selected task to the terminal for processing, further comprising the step of: detecting whether the preset is The task processing result is received within the duration, and if not, the selected task is reassigned to other terminals that send the task processing request for processing.

在一个实施例中，当该计算机可读指令被一个或多个处理器执行时，使得该处理器还执行以下步骤：统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。In one embodiment, when the computer readable instructions are executed by one or more processors, the processor is further caused to perform the steps of counting the number of tasks completed by each terminal and sorting the terminals according to the number of completed tasks.

在一个实施例中，提供了一种应用于分布式任务调度***中的服务器，包括存储器及处理器，存储器中储存有计算机可读指令，该分布式任务调度***包括所述服务器和多个终端，该服务器为所述分布式任务调度***中的主控节点，每个终端为所述分布式任务调度***中的执行节点，该指令被处理器执行时，使得处理器执行上述各个实施例所提供的任务分布式处理方法的步骤。In one embodiment, a server is provided for use in a distributed task scheduling system, including a memory and a processor, the memory storing computer readable instructions, the distributed task scheduling system including the server and a plurality of terminals The server is a master node in the distributed task scheduling system, and each terminal is an execution node in the distributed task scheduling system, and the instruction is When the processor executes, the processor is caused to perform the steps of the task distributed processing method provided by the various embodiments described above.

在一个实施例中，该指令被处理器执行时，使得处理器执行以下步骤：接收终端发送的任务处理请求；选取至少一个待分配状态的任务，将选取的任务分配给终端进行处理；接收终端反馈的任务处理结果；根据任务处理结果修改对应任务的任务状态。In one embodiment, when the instruction is executed by the processor, the processor is configured to: receive a task processing request sent by the terminal; select at least one task to be assigned, assign the selected task to the terminal for processing; and receive the terminal Feedback task processing result; modify the task status of the corresponding task according to the task processing result.

在一个实施例中，如图7所示，为一个实施例中服务器的内部结构示意图。该服务器包括通过***总线连接的包括通过***总线连接的处理器、存储器和网络接口。其中，该服务器的处理器用于提供计算和控制能力，支撑整个终端的运行。存储器用于存储数据、指令代码等，网络接口用于与终端进行网络通信。比如，可向终端发送所选取的任务等。存储器上存储至少一个计算机可读指令，该计算机可读指令可被处理器执行，以实现本申请实施例中提供的适用于服务器的任务分布式处理方法。存储器可包括磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)等非易失性存储介质等。例如，在一个实施例中，存储器包括非易失性存储介质及内存储器。服务器的非易失性存储介质存储有操作***和计算机可读指令。该计算机可读指令可被处理器所执行，以用于实现以上各个实施例所提供的任务分布式处理方法。终端中的内存储器为非易失性存储介质中的操作***、数据库和计算机可读指令提供高速缓存的运行环境。网络接口可以是以太网卡或无线网卡等，用于与外部的终端或服务器进行通信。服务器可以用独立的服务器或者是多个服务器组成的服务器集群来实现。本领域技术人员可以理解，图7中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的服务器的限定，具体的服务器可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。In one embodiment, as shown in FIG. 7, is an internal structure diagram of a server in one embodiment. The server includes a processor, memory, and network interface connected through a system bus including connections through a system bus. The processor of the server is used to provide computing and control capabilities to support the operation of the entire terminal. The memory is used to store data, instruction codes, etc., and the network interface is used for network communication with the terminal. For example, the selected task or the like can be sent to the terminal. At least one computer readable instruction is stored on the memory, and the computer readable instructions are executable by the processor to implement the task-distributed processing method for the server provided in the embodiments of the present application. The memory may include a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or the like. For example, in one embodiment, the memory includes a non-volatile storage medium and an internal memory. The server's non-volatile storage medium stores operating system and computer readable instructions. The computer readable instructions are executable by a processor for implementing the task distributed processing methods provided by the various embodiments above. The internal memory in the terminal provides a cached operating environment for operating systems, databases, and computer readable instructions in a non-volatile storage medium. The network interface may be an Ethernet card or a wireless network card or the like for communicating with an external terminal or server. The server can be implemented with a stand-alone server or a server cluster consisting of multiple servers. It will be understood by those skilled in the art that the structure shown in FIG. 7 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. The specific server may include a ratio. More or fewer components are shown in the figures, or some components are combined, or have different component arrangements.

在一个实施例中，该指令被处理器执行时，使得处理器执行的选取至少一个待分配状态的任务，包括：选取至少一个待分配状态的任务标识；从主数据库中读取每个任务标识对应的任务，当从主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。In one embodiment, when the instruction is executed by the processor, causing the processor to perform the task of selecting at least one state to be allocated, including: selecting at least one task identifier to be assigned; reading each task identifier from the primary database Corresponding task, when reading from the primary database fails, from standby The task corresponding to each task ID is read in the database.

在一个实施例中，任务处理请求中携带终端的互联网协议地址；该指令被处理器执行时，使得处理器执行的将选取的任务分配给终端进行处理，包括：根据互联网协议地址，将选取的任务分配给终端进行处理。In an embodiment, the task processing request carries the Internet Protocol address of the terminal; when the instruction is executed by the processor, the processor performs the process of assigning the selected task to the terminal for processing, including: selecting the selected according to the Internet Protocol address. Tasks are assigned to the terminal for processing.

在一个实施例中，该指令被处理器执行时，使得处理器在执行将选取的任务分配给终端进行处理之后，还包括执行以下步骤：接收终端发送的心跳包，以根据心跳包确定终端正在处理选取的任务。In an embodiment, when the instruction is executed by the processor, after the processor performs the process of assigning the selected task to the terminal for processing, the method further includes the step of: receiving a heartbeat packet sent by the terminal, to determine, according to the heartbeat packet, that the terminal is Process selected tasks.

在一个实施例中，该指令被处理器执行时，使得处理器在执行将选取的任务分配给终端进行处理之后，还包括执行以下步骤：检测是否在预设时长之内接收到任务处理结果，若否，则将选取的任务重新分配给其它发送任务处理请求的终端进行处理。In an embodiment, when the instruction is executed by the processor, after the processor performs the process of assigning the selected task to the terminal for processing, the method further includes the step of: detecting whether the task processing result is received within the preset duration, If not, the selected task is reassigned to other terminals that send the task processing request for processing.

在一个实施例中，该指令被处理器执行时，使得处理器还执行以下步骤：统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。In one embodiment, when the instructions are executed by the processor, the processor further performs the steps of: counting the number of tasks completed by each terminal, and ordering the terminals according to the number of completed tasks.

本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)等。One of ordinary skill in the art can understand that all or part of the process of implementing the above embodiments can be completed by computer readable instructions, which can be stored in a non-volatile computer. The readable storage medium, which when executed, may include the flow of an embodiment of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or the like.

以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。The technical features of the above-described embodiments may be arbitrarily combined. For the sake of brevity of description, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, All should be considered as the scope of this manual.

以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。 The above-mentioned embodiments are merely illustrative of several embodiments of the present application, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the scope of the invention should be determined by the appended claims.

Claims

一种任务分布式处理方法，应用于分布式任务调度***中，其特征在于，所述分布式任务调度***包括服务器和多个终端，所述服务器为所述分布式任务调度***中的主控节点，每个所述终端为所述分布式任务调度***中的执行节点；所述方法包括：A task distributed processing method is applied to a distributed task scheduling system, wherein the distributed task scheduling system includes a server and a plurality of terminals, and the server is a master in the distributed task scheduling system. a node, each of the terminals being an execution node in the distributed task scheduling system; the method includes:

所述服务器接收所述终端发送的任务处理请求；Receiving, by the server, a task processing request sent by the terminal;

选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；Selecting at least one task to be assigned, and assigning the selected task to the terminal for processing;

接收所述终端反馈的任务处理结果；及Receiving a task processing result fed back by the terminal; and

根据所述任务处理结果修改对应任务的任务状态。Modifying the task status of the corresponding task according to the task processing result.
根据权利要求1所述的方法，其特征在于，所述选取至少一个待分配状态的任务，包括：The method according to claim 1, wherein the selecting at least one task to be assigned includes:

选取至少一个待分配状态的任务标识；及Select at least one task ID to be assigned status; and

从主数据库中读取每个任务标识对应的任务，当从所述主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。The task corresponding to each task identifier is read from the primary database, and when the read from the primary database fails, the task corresponding to each task identifier is read from the standby database.
根据权利要求1所述的方法，其特征在于，所述任务处理请求中携带所述终端的互联网协议地址；The method according to claim 1, wherein the task processing request carries an internet protocol address of the terminal;

所述将选取的任务分配给所述终端进行处理，包括：The assigning the selected task to the terminal for processing, including:

根据所述互联网协议地址，将选取的任务分配给所述终端进行处理。Assigning the selected task to the terminal for processing according to the Internet Protocol address.
根据权利要求1所述的方法，其特征在于，在所述将选取的任务分配给所述终端进行处理之后，包括：The method according to claim 1, wherein after the selecting the selected task to the terminal for processing, the method comprises:

接收所述终端发送的心跳包，以根据所述心跳包确定所述终端正在处理所述选取的任务。Receiving a heartbeat packet sent by the terminal, to determine, according to the heartbeat packet, that the terminal is processing the selected task.
根据权利要求1所述的方法，其特征在于，在所述将选取的任务分配给所述终端进行处理之后，包括：The method according to claim 1, wherein after the selecting the selected task to the terminal for processing, the method comprises:

检测是否在预设时长之内接收到所述任务处理结果，若否，则将所述选取的任务重新分配给其它发送任务处理请求的终端进行处理。 It is detected whether the task processing result is received within a preset duration, and if not, the selected task is re-allocated to other terminals that send the task processing request for processing.
根据权利要求1所述的方法，其特征在于，所述方法还包括：The method of claim 1 further comprising:

统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。The number of tasks completed by each terminal is counted, and the terminals are sorted according to the number of completed tasks.
一种应用于分布式任务调度***中的任务分布式处理装置，包括：A task distributed processing device applied to a distributed task scheduling system, comprising:

请求接收模块，用于接收终端发送的任务处理请求，所述终端为所述分布式任务调度***中的执行节点；a request receiving module, configured to receive a task processing request sent by the terminal, where the terminal is an execution node in the distributed task scheduling system;

任务分配模块，用于选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；及a task assignment module, configured to select at least one task to be assigned, and assign the selected task to the terminal for processing; and

任务状态修改模块，用于接收所述终端反馈的任务处理结果；根据所述任务处理结果修改对应任务的任务状态。The task status modification module is configured to receive a task processing result fed back by the terminal, and modify a task status of the corresponding task according to the task processing result.
根据权利要求7所述的装置，其特征在于，所述任务分配模块还用于选取至少一个待分配状态的任务标识；从主数据库中读取每个任务标识对应的任务，当从所述主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。The device according to claim 7, wherein the task assignment module is further configured to: select at least one task identifier to be assigned; and read a task corresponding to each task identifier from the primary database, when from the primary When the database read fails, the task corresponding to each task ID is read from the standby database.
一个或多个存储有计算机可读指令的计算机可读非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行一种任务分布式处理方法，所述方法应用于分布式任务调度***中，所述分布式任务调度***包括服务器和多个终端，所述服务器为所述分布式任务调度***中的主控节点，每个所述终端为所述分布式任务调度***中的执行节点；所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：One or more computer readable non-volatile storage media storing computer readable instructions, when executed by one or more processors, causing the one or more processors to perform a task a distributed processing method, the method being applied to a distributed task scheduling system, the distributed task scheduling system comprising a server and a plurality of terminals, wherein the server is a master node in the distributed task scheduling system, each The terminal is an execution node in the distributed task scheduling system; when the computer readable instructions are executed by one or more processors, the one or more processors perform the following steps:

接收所述终端发送的任务处理请求；Receiving a task processing request sent by the terminal;

选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；Selecting at least one task to be assigned, and assigning the selected task to the terminal for processing;

接收所述终端反馈的任务处理结果；及Receiving a task processing result fed back by the terminal; and

根据所述任务处理结果修改对应任务的任务状态。Modifying the task status of the corresponding task according to the task processing result.
根据权利要求9所述的存储介质，其特征在于，所述计算机可读指令被一个或多个处理器执行时，还使得所述一个或多个处理器执行以下步骤： The storage medium of claim 9, wherein when the computer readable instructions are executed by one or more processors, the one or more processors are further caused to:

选取至少一个待分配状态的任务标识；Select at least one task identifier to be assigned status;

从主数据库中读取每个任务标识对应的任务，当从所述主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。The task corresponding to each task identifier is read from the primary database, and when the read from the primary database fails, the task corresponding to each task identifier is read from the standby database.
根据权利要求9所述的存储介质，其特征在于，所述任务处理请求中携带所述终端的互联网协议地址；The storage medium according to claim 9, wherein the task processing request carries an internet protocol address of the terminal;

所述计算机可读指令被一个或多个处理器执行时，还使得所述一个或多个处理器执行以下步骤：根据所述互联网协议地址，将选取的任务分配给所述终端进行处理。The computer readable instructions, when executed by one or more processors, further cause the one or more processors to perform the step of assigning selected tasks to the terminal for processing in accordance with the internet protocol address.
根据权利要求9所述的存储介质，其特征在于，所述计算机可读指令被一个或多个处理器执行时，还使得所述一个或多个处理器执行以下步骤：The storage medium of claim 9, wherein when the computer readable instructions are executed by one or more processors, the one or more processors are further caused to:

接收所述终端发送的心跳包，以根据所述心跳包确定所述终端正在处理所述选取的任务。Receiving a heartbeat packet sent by the terminal, to determine, according to the heartbeat packet, that the terminal is processing the selected task.
根据权利要求9所述的存储介质，其特征在于，所述计算机可读指令被一个或多个处理器执行时，还使得所述一个或多个处理器执行以下步骤：The storage medium of claim 9, wherein when the computer readable instructions are executed by one or more processors, the one or more processors are further caused to:

检测是否在预设时长之内接收到所述任务处理结果，若否，则将所述选取的任务重新分配给其它发送任务处理请求的终端进行处理。It is detected whether the task processing result is received within a preset duration, and if not, the selected task is re-allocated to other terminals that send the task processing request for processing.
根据权利要求9所述的存储介质，其特征在于，所述计算机可读指令被一个或多个处理器执行时，还使得所述一个或多个处理器执行以下步骤：The storage medium of claim 9, wherein when the computer readable instructions are executed by one or more processors, the one or more processors are further caused to:

统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。The number of tasks completed by each terminal is counted, and the terminals are sorted according to the number of completed tasks.
一种应用于分布式任务调度***中的服务器，包括存储器及处理器，所述存储器中储存有计算机可读指令，所述分布式任务调度***包括所述服务器和多个终端，所述服务器为所述分布式任务调度***中的主控节点，每个所述终端为所述分布式任务调度***中的执行节点；所述指令被所述处理器执行时，使得所述处理器执行以下步骤：A server for use in a distributed task scheduling system includes a memory and a processor, wherein the memory stores computer readable instructions, the distributed task scheduling system includes the server and a plurality of terminals, and the server is a master node in the distributed task scheduling system, each of the terminals being an execution node in the distributed task scheduling system; when the instructions are executed by the processor, causing the processor to perform the following steps :

接收所述终端发送的任务处理请求；Receiving a task processing request sent by the terminal;

选取至少一个待分配状态的任务，将选取的任务分配给所述终端进行处理；Selecting at least one task to be assigned, and assigning the selected task to the terminal for performing Reason

接收所述终端反馈的任务处理结果；及Receiving a task processing result fed back by the terminal; and

根据所述任务处理结果修改对应任务的任务状态。Modifying the task status of the corresponding task according to the task processing result.
根据权利要求15所述的处理器，其特征在于，所述指令被所述处理器执行时，还使得所述处理器执行以下步骤：The processor of claim 15 wherein said instructions, when executed by said processor, further cause said processor to perform the following steps:

选取至少一个待分配状态的任务标识；及Select at least one task ID to be assigned status; and

从主数据库中读取每个任务标识对应的任务，当从所述主数据库读取失败时，从备用数据库中读取每个任务标识对应的任务。The task corresponding to each task identifier is read from the primary database, and when the read from the primary database fails, the task corresponding to each task identifier is read from the standby database.
根据权利要求15所述的处理器，其特征在于，所述任务处理请求中携带所述终端的互联网协议地址；The processor according to claim 15, wherein the task processing request carries an internet protocol address of the terminal;

所述指令被所述处理器执行时，还使得所述处理器执行以下步骤：When the instructions are executed by the processor, the processor is further caused to perform the following steps:

根据所述互联网协议地址，将选取的任务分配给所述终端进行处理。Assigning the selected task to the terminal for processing according to the Internet Protocol address.
根据权利要求15所述的处理器，其特征在于，所述指令被所述处理器执行时，还使得所述处理器执行以下步骤：The processor of claim 15 wherein said instructions, when executed by said processor, further cause said processor to perform the following steps:

接收所述终端发送的心跳包，以根据所述心跳包确定所述终端正在处理所述选取的任务。Receiving a heartbeat packet sent by the terminal, to determine, according to the heartbeat packet, that the terminal is processing the selected task.
根据权利要求15所述的处理器，其特征在于，所述指令被所述处理器执行时，还使得所述处理器执行以下步骤：The processor of claim 15 wherein said instructions, when executed by said processor, further cause said processor to perform the following steps:

检测是否在预设时长之内接收到所述任务处理结果，若否，则将所述选取的任务重新分配给其它发送任务处理请求的终端进行处理。It is detected whether the task processing result is received within a preset duration, and if not, the selected task is re-allocated to other terminals that send the task processing request for processing.
根据权利要求15所述的处理器，其特征在于，所述指令被所述处理器执行时，还使得所述处理器执行以下步骤：The processor of claim 15 wherein said instructions, when executed by said processor, further cause said processor to perform the following steps:

统计每个终端完成的任务数量，根据完成的任务数量对终端进行排序。 The number of tasks completed by each terminal is counted, and the terminals are sorted according to the number of completed tasks.