WO2020215752A1 - 图计算方法及装置 - Google Patents

图计算方法及装置 Download PDF

Info

Publication number
WO2020215752A1
WO2020215752A1 PCT/CN2019/125798 CN2019125798W WO2020215752A1 WO 2020215752 A1 WO2020215752 A1 WO 2020215752A1 CN 2019125798 W CN2019125798 W CN 2019125798W WO 2020215752 A1 WO2020215752 A1 WO 2020215752A1
Authority
WO
WIPO (PCT)
Prior art keywords
graph
algorithm
task
computing
subgraph
Prior art date
Application number
PCT/CN2019/125798
Other languages
English (en)
French (fr)
Inventor
夏应龙
张晨逸
方扬
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP19925708.0A priority Critical patent/EP3955132A4/en
Publication of WO2020215752A1 publication Critical patent/WO2020215752A1/zh
Priority to US17/508,294 priority patent/US20220043675A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9024—Graphs; Linked lists
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • This application relates to the field of computer technology, and in particular to a graph calculation method and device.
  • a graph is a basic data structure that contains a series of nodes and edges connecting the nodes. Many entities in life and the relationship between entities can be expressed intuitively through the points and edges on the graph. Therefore, graph-based analysis technology-graph computing technology came into being.
  • the application of graph computing technology can integrate social networks and call networks. , The bipartite graph between users and products, the cooperative relationship network between the authors in the paper, the index relationship between the articles, the financial transaction network, the interaction between wireless base stations in a region and various servers in the cloud, and mobile game users
  • Various practical application scenarios such as the data exchange relationship of the data exchange are modeled as graphs, and the laws contained in the entities represented by the graphs are mined by calculating the graphs.
  • computing devices will receive batch calculation requests.
  • the request is used to request that one or more algorithms be used to calculate the graph.
  • the computing device will use each algorithm to calculate the graph in a serial manner. Specifically, assuming that the calculation request requires the use of N algorithms to calculate a certain graph, the computing device will first load the graph from the external memory to the memory, and use Algorithm 1 to calculate the graph.
  • the present application provides a graph calculation method and device, which can improve the calculation efficiency of graph calculation.
  • the first aspect of the present application provides a graph computing method, which may not be limited to a graph computing device.
  • the graph computing device may be multiple computing units in a distributed cloud environment.
  • the method includes:
  • At least one calculation request being used to request that multiple algorithms be used to calculate the graph
  • the multiple algorithms are executed in parallel.
  • the above provides a method that can support the concurrent execution of graph calculations by multiple algorithms.
  • multiple calculation units can execute multiple algorithms in parallel, which can be shared among multiple algorithms
  • multiple algorithms can be executed in parallel on the same graph, which saves the time delay caused by waiting for the execution of other algorithms when executing an algorithm, thereby improving the overall efficiency of graph calculation by multiple algorithms and shortening the number of algorithms.
  • the overall time for the algorithm to calculate the graph is not limited to:
  • the graphs loaded in the memory can be reused when executing graph calculations by multiple algorithms, instead of loading the graph into the memory and releasing the graph from the memory for each algorithm. This avoids the overhead caused by repeatedly loading and releasing the same image into memory, saving time for accessing memory, and breaking input/output (full English name: input/output, English abbreviation: IO )bottleneck.
  • the parallel execution of the multiple algorithms by the multiple computing units includes:
  • the acquiring at least one task in each algorithm includes at least one of the following:
  • the algorithm is divided into collection tasks, application tasks, and distribution tasks;
  • the algorithm is divided into local tasks, communication tasks and synchronization tasks;
  • the method provided above by decomposing the algorithm into tasks, can group tasks with the same data access method for different algorithms dealing with the same graph, breaking the barriers between different algorithms, and effectively exposing the tasks of each task.
  • the similarity of data access patterns so that the same type of tasks have similar regularity in accessing graphs, and the tasks are scheduled and allocated reasonably, so as to rationally use and schedule system resources and provide higher overall performance. service.
  • it is used in the concurrent execution of multiple users and multiple algorithms.
  • By decomposing multiple algorithms into tasks it is helpful for unified management of multiple algorithms.
  • the loading multiple subgraphs of the graph to multiple calculation units includes:
  • the parallel execution of tasks in the multiple algorithms by the multiple computing units includes:
  • the task is executed according to the subgraph of the target mode through the plurality of calculation units, and the target mode is matched with the task in the at least one mode Modal.
  • the task includes a step of searching from the first vertex in the graph to the second vertex in the graph, and the second vertex in the subgraph of the target mode is arranged in all Before the first vertex.
  • the method provided above considers the close relationship between different types of tasks and subgraphs of different modes, and realizes the multi-modal management of the graph. It can associate the assignment of tasks with the modalities of the subgraphs, thereby linking tasks Assigned to the calculation unit loaded with the appropriate subgraph. It can provide a relatively optimized data structure for the task, and improve the problem of limited data locality that is common in the field of graph computing, thereby increasing the execution speed of the computing task.
  • the collection task needs to access the preorder information of each vertex, and the subgraph of the in-edge mode just aggregates the preorder information of each vertex together, the data locality can be improved; similarly, For the distribution task, the distribution task needs to provide information to all subsequent nodes, then the distribution task is allocated to the computing unit loaded with the out-of-edge mode for processing, which can improve the locality of the data and increase the computational efficiency.
  • the parallel execution of tasks in the multiple algorithms by the multiple computing units includes:
  • a scheduling scheme is obtained, the scheduling scheme is used to indicate the correspondence between at least one target task and at least one target subgraph, and the target task is the current time among the tasks of the multiple algorithms.
  • the target sub-picture is a sub-picture of the sub-picture currently scheduled in the multiple sub-pictures;
  • the at least one target task is executed in parallel by multiple computing units loaded with the at least one target subgraph.
  • the parallel execution of tasks in the multiple algorithms by the multiple computing units includes:
  • a scheduling scheme is obtained, the scheduling scheme is used to indicate the correspondence between at least one target task and at least one target subgraph, and the target task is the current time among the tasks of the multiple algorithms.
  • the target sub-picture is a sub-picture of the sub-picture currently scheduled in the multiple sub-pictures;
  • the at least one target task is executed in parallel by multiple computing units loaded with the at least one target subgraph.
  • the methods provided above can ensure that the method of defining the priority of the algorithm can be customized by the user to meet user needs. That is, it is beneficial for the system to implement scheduling according to different scheduling goals, so that the system has good configurability.
  • the parallel execution of the multiple algorithms by the multiple computing units includes at least one of the following:
  • the loading step is a step of loading the graph into the memory of the computing unit
  • the release step is a step of releasing the graph from the memory of the computing unit.
  • the process of releasing the subgraph from the memory of the computing unit also saves the time and performance overhead caused by repeatedly releasing the same subgraph.
  • the method before the receiving at least one calculation request, the method further includes:
  • the loading multiple sub-graphs of the graph to at least one multiple computing units includes:
  • the multiple sub-graphs of the graph are loaded from the graph storage device to the multiple calculation units.
  • multiple subgraphs can be directly loaded into the multiple calculation units without temporarily dividing the graph into subgraphs. It also saves the time delay caused by temporarily dividing the graph into sub-graphs, can speed up the execution of multiple algorithms, improve the efficiency of executing multiple algorithms, and help to quickly process and complete calculation requests.
  • the dividing the graph includes:
  • the graph is divided according to the number of the multiple calculation units.
  • the method provided above can support the average loading of subgraphs to multiple computing units, that is, when multiple subgraphs of the graph are loaded into the computing unit, the number of subgraphs loaded by different computing units is the same or approximately the same, thereby ensuring that each When the computing unit executes the algorithm according to the loaded subgraph, the load is balanced among different computing units.
  • the method further includes:
  • the multiple algorithms are executed in parallel.
  • the capacity expansion request includes at least one of the number of newly added computing units and the capacity expansion cost.
  • the method provided above can automatically perform dynamic expansion during the parallel execution of multiple algorithms on the graph.
  • the overall computing power of the system can be improved, thereby reducing the delay in executing multiple algorithms .
  • the load balance of the system can be realized by offloading the workload required to execute the algorithm to the newly added computing unit.
  • the adaptive capability of the system can be improved.
  • the copying at least one sub-graph of the graph includes:
  • the methods provided above take into account that the number of accesses that a single graph can support has a physical upper limit. If the number of accesses to a graph exceeds the physical upper limit, it is likely that the graph cannot support parallel execution of multiple algorithms, which will affect the execution process of the graph algorithm. bottleneck. For example, if a certain subgraph is a hotspot subgraph, for example, the subgraph is a subgraph that contains celebrity information, and it is likely that a large number of users visit the subgraph to query celebrity information at the same time, the subgraph may not be able to support being When multiple tasks are scheduled at the same time, multiple algorithms cannot be executed in parallel, which limits the overall speed of graph calculation. In the above implementation, the graph computing device can perceive the number of requests for each sub-graph.
  • the number of requests for a certain sub-graph exceeds the threshold, it indicates that the demand for the sub-graph is very large, and it is likely to be a hot sub-graph.
  • the subgraph is copied into multiple copies and deployed on different computing units. Multiple computing units process the subgraph, which can improve the processing efficiency of the subgraph. That is, by allowing concurrent computing requests to be diverted to different instances for computing, the concurrency is linearly expanded.
  • the second aspect of the present application provides a graph computing device configured to execute the first aspect or the method provided by any possible implementation manner of the first aspect.
  • the graph computing device includes units for executing the first aspect or the method provided by any possible implementation of the first aspect.
  • the third aspect of the present application provides a graph computing device, including multiple computing units, each computing unit includes a processor and a memory, the memory stores at least one instruction, the instruction is loaded and executed by the processor A method provided to implement the first aspect or any possible implementation manner of the first aspect.
  • the fourth aspect of the present application provides a graph computing system, including a graph computing device and a graph storage device;
  • the graph computing device includes multiple computing units
  • the graph storage device is used to store multiple subgraphs of the graph
  • the graph computing device is configured to load the multiple sub-graphs from the graph storage device to the multiple computing units to execute the first aspect or the method provided by any possible implementation of the first aspect.
  • the fifth aspect of the present application provides a non-transitory readable storage medium.
  • the graph computing device executes the aforementioned first aspect or first aspect.
  • the storage medium stores the program.
  • the type of the storage medium includes but is not limited to volatile memory, such as random access memory, non-volatile memory, such as flash memory, hard disk drive (HDD), solid state drive (SSD).
  • the sixth aspect of the present application provides a computer program product.
  • the graph computing device executes the aforementioned first aspect or the method provided in any possible implementation of the first aspect .
  • the computer program product may be a software installation package.
  • the computer program product may be downloaded and executed on a graph computing device.
  • FIG. 1 is an architecture diagram of a graph computing system provided by an embodiment of the application
  • FIG. 2 is a functional architecture diagram of a graph computing device provided by an embodiment of this application.
  • FIG. 3 is a working flowchart of a graph computing device provided by an embodiment of the application.
  • FIG. 4 is another working flowchart of the graph computing device provided by an embodiment of the application.
  • FIG. 5 is another working flowchart of the graph computing device provided by the embodiment of the application.
  • FIG. 6 is a schematic flow diagram of the method provided by this application.
  • FIG. 7 is a schematic diagram of the sub-graph of the in-edge mode and the sub-graph of the out-edge mode provided by this application;
  • Figure 8 is a schematic diagram of algorithm decomposition and unified management of multiple algorithms provided by this application.
  • FIG. 9 is a schematic diagram of task scheduling with multiple algorithms provided by this application.
  • FIG. 10 is a schematic diagram of task scheduling based on sub-graph mode provided by this application.
  • FIG. 11 is a schematic diagram of a first bipartite graph and a second bipartite graph provided by this application;
  • FIG. 12 is a flowchart of an iterative algorithm provided by this application.
  • FIG. 13 is a schematic structural diagram of a graph computing device provided by this application.
  • FIG. 16 is a schematic structural diagram of the graph computing system provided by this application.
  • unit in this application can be implemented by hardware, or by hardware executing corresponding software, and the hardware or software has the function of executing corresponding steps in the following methods.
  • the receiving unit may be replaced by a transceiver to execute the receiving steps in the following method
  • the computing unit may be replaced by a processor and a memory to execute the executing steps in the following method.
  • module in this application may be a software module.
  • Graph A data structure including at least one vertex and at least one edge.
  • vertices in the graph can be mapped to entities, and edges in the graph can be mapped to relationships between entities and entities.
  • the graph can be a directed graph or an undirected graph.
  • the graph can also include other data besides vertices and edges, such as vertex labels and edge labels.
  • each vertex in the graph in a friend recommendation scenario, can represent a user, and each edge in the graph can represent the social relationship between different users.
  • the data of each vertex in the graph is User portrait data and user behavior data, such as the user’s age, occupation, hobbies, education, etc.
  • each vertex in the graph when applied to a commodity recommendation scenario, can represent a user or a commodity, and each edge in the graph can represent the interaction relationship between the user and the commodity, such as purchase relationship, collection relationship, etc. .
  • each vertex in the graph in the scenario of financial risk control, can represent an account, transaction, or capital.
  • the edges in the figure can represent the flow of funds.
  • the loop in the figure can represent cyclic transfers.
  • each vertex in the graph can represent a network element, such as routers, switches, terminals, etc., and each edge in the graph can represent the connection relationship between different network elements.
  • Algorithm Including a series of operations on the vertices and/or edges of the graph to perform statistics, sorting, path selection and other operations on the vertices and/or edges of the graph.
  • the algorithm can be a webpage ranking algorithm (also known as PageRank algorithm), Pixie random walk algorithm, breadth-first search (English full name: Breadth-First Search, English abbreviation: BFS) algorithm, depth first search (English full name: Depth- First-Search) algorithm, personalized web page ranking algorithm (also known as PersonalRank algorithm), k-core (English: k-core) algorithm, k-hop (English: k-hop) algorithm, shortest path (English: shortest paths) algorithm, All shortest paths (English: all shortest paths) algorithm, associated path algorithm, tight centrality algorithm, label propagation algorithm, community discovery algorithm based on modularity (English: Louvain algorithm), association prediction algorithm, node2vec algorithm (a kind of network The algorithm in which nodes are mapped to Euclidean space), real-time recommendation algorithm
  • Subgraph is a part of a graph, including some vertices and some edges in the graph.
  • the sub-picture may also be called a partition (English: partition) in the picture.
  • a graph can include multiple subgraphs.
  • Collection application distribution (English full name: gather apply scatter, English abbreviation: GAS) model: a programming specification for an algorithm, according to the GAS model, the algorithm can be divided into three tasks, namely collection (English: gather) task, application ( English: apply) tasks and distribution (English: scatter) tasks.
  • the collection task can include the steps of obtaining data from the neighboring vertices of the vertices in the graph and the step of obtaining the data of the edges in the graph.
  • the collection task can be regarded as the process of collecting data from the neighboring vertices and itself by the working vertex.
  • the collection task can be Summing the data of each edge in the graph, another example, the collection task can be to calculate the number of vertices adjacent to a vertex.
  • the points and edges can be read-only.
  • the application task may include the step of updating the data of the vertices in the graph and the step of synchronizing the data of the vertices in the graph to the mirror vertices of the vertices.
  • the application task may be the number of adjacent vertices of the same vertex calculated by each calculation vertex.
  • the application task can be that the mirror vertex sends the calculation result of the collection task to the main vertex.
  • the main vertex summarizes the calculation results of multiple mirror vertices, and uses the summary result and the vertex data of the previous step to perform further calculations according to business needs, and then update the main vertex
  • the data is synchronized to the mirror vertex.
  • working vertices can be in a modifiable state, and edges can be in an unmodifiable state.
  • the distribution task may include the step of sending data to adjacent vertices of the vertices in the graph and the step of updating the data of the edges in the graph.
  • the distribution task may be a process of updating the data on the edge after the work vertex completes the data update, and notifying the neighboring vertices that are dependent on the work vertex to update the status.
  • the working vertices can be read-only, and the edges can be writable.
  • Local tasks also called local computing (English: local computing) tasks may include calculation steps in an algorithm based on local data.
  • a local task may include a calculation process that a single processing unit needs to complete.
  • the communication (English: communication) task may include steps in the algorithm that require different computing units to interact.
  • the communication task may include a step in the algorithm in which the processing unit A sends the calculation result of the subgraph to the processing unit B.
  • the synchronization task may include the step of waiting for the completion of the communication task in the algorithm.
  • the synchronization task may include the process of barrier synchronization.
  • Modality refers to the form of data, or the way of expression of data. For the same data, the different modes of the data are different in form, but the essential information contained is the same.
  • the subgraph may have at least one mode. For example, for a certain subgraph, the vertices contained in the subgraph of mode 1 and the vertices contained in the subgraph of mode 2 are the same, and the edges of the subgraph of mode 1 are the same as those of mode 2. The edges contained in the subgraph are the same, and the arrangement order of the vertices contained in the subgraph of mode 1 and the arrangement order of the vertices contained in the subgraph of mode 2 may be different.
  • the subgraph of mode 1 and the subgraph of mode 2 both contain vertex 1, vertex 2...vertex 100, and the subgraph of mode 1
  • the vertex 1 in the subgraph is arranged in front of the vertex 2, and the vertex 1 in the subgraph of the mode 2 is arranged behind the vertex 2.
  • In-edge mode refers to the mode of the subgraph constructed based on the in-edge.
  • the in-edge mode is used to indicate the in-edge relationship between different vertices in the sub-graph. Specifically, the end point of each edge in the subgraph can be ranked first, and the start point of each edge can be ranked back.
  • vertex A in the subgraph of the in-edge mode can be ranked first, and vertex B and vertex C can be ranked behind.
  • Out-edge mode refers to the mode of the sub-graph constructed based on the out-edge.
  • the sub-graph of the out-edge mode can include at least one out-edge. Specifically, the starting point of each edge in the sub-graph can be ranked first, and each The end point of the edge can be arranged later.
  • vertex A For example, if the subgraph includes vertex A, vertex B, and vertex C, the relationship between these three vertices is: there is an edge between vertex A and vertex B, and the direction of the edge is that vertex A points to vertex B, and vertex A There is another edge with vertex C, and the direction of the edge is that vertex A points to vertex C, then vertex B and vertex C in the subgraph of the out-edge mode can be ranked first, and vertex A can be ranked behind.
  • the graph computing system includes a graph computing device 101 and a terminal 102.
  • the terminal 102 is used to send a computing request to the graph computing device 101, and the graph computing device 101 is used for The calculation request of the terminal 102 is received, the graph calculation method provided in the following method embodiment is executed, and the calculation result is sent to the terminal 102.
  • the graph computing device 101 may include one or more computing units, and different computing units may execute the same or different steps in the following method embodiments.
  • the computing unit may be a physical device; for example, the computing unit may be a physical server, a central processing unit (full English name: central processing unit, English abbreviation: CPU), etc.
  • the computing unit may be a virtualized device; for example, the computing unit may be a virtual machine, a container, or a pod (pod refers to the basic unit of container operation when running, managing, and orchestrating containerized applications in Kubernetes) , The core of the processor, and so on.
  • the computing unit may be a basic unit in software.
  • the computing unit may be an application, service, microservice, module, sub-module, class, or function.
  • the specifications of one or more computing units in the graph computing device 101 may be the same.
  • one or more computing units may include the same number of processors and the same capacity of memory.
  • the specifications of one or more computing units may also be the same. There are differences, which are not limited in this embodiment.
  • Multiple computing units can be realized by a single machine, or can form a distributed system.
  • multiple computing units may run on the same physical device, and different computing units may communicate through the communication network inside the physical device.
  • multiple computing units can be multiple virtual machines or multiple containers in a physical server; for another example, multiple computing units can be multiple processors or the same processing in a physical server that communicate through a bus. Multiple cores in the processor.
  • different computing units in the graph computing device 101 can run on different physical devices, for example, they can run in different locations, different computing centers, different computer rooms, and different racks. Different physical devices in the computing device 101 can interact through the network.
  • the graph computing device 101 can be provided to users as a cloud computing service, for example, can be provided to users as a graph engine service (full English name: Graph Engine Service, English abbreviation: GES).
  • the graph computing device 101 may run in a cloud environment, for example, it may run on a public cloud, a private cloud, or a hybrid cloud.
  • the graph computing device 101 may be an elastic cloud server (English full name: elastic cloud server, English abbreviation: ECS) cluster, and each computing unit in the graph computing device 101 is an ECS; for another example, the graph computing device 101 may be Virtual machine cluster, each computing unit in the graph computing device 101 is a virtual machine running in a cloud environment; another example, the graph computing device 101 can be provided as a cloud container engine (full English name: cloud container engine, English abbreviation: CCE ), each computing unit in the graph computing device 101 is a container running in a cloud environment; for another example, the graph computing device 101 can be provided as a cloud service stage (English: cloud service stage), and each computing device in the graph computing device 101 A computing unit is an application, service or microservice running in a cloud environment.
  • ECS elastic cloud server
  • the graph computing device 101 running in a cloud environment is only an example, the graph computing device 101 can also run in an edge environment, and each computing unit in the graph computing device 101 can be an edge computing device in the edge environment; The apparatus 101 may also run in an edge environment, and each computing unit in the graph computing apparatus 101 may be a terminal device in a terminal environment; this embodiment does not limit the operating environment of the graph computing apparatus 101.
  • each computing unit of the graph computing device 101 may also run in different environments.
  • the graph computing device 101 can run part of the computing units of the graph computing device 101 in three of the cloud environment, the edge environment, and the terminal environment, or any two of them.
  • the terminal 102 may be a mobile phone, a notebook, a server, a desktop computer, etc.
  • the terminal 102 can interact with the graph computing device 101 via a network.
  • the graph computing system may further include a graph storage device 103, and the graph storage device 103 is configured to provide the graph computing device 101 with a service of storing graphs.
  • the graph storage device 103 can be realized by a single machine, or can form a distributed system.
  • the image storage device 103 may be a distributed memory.
  • the graph storage device 103 can be implemented through a cloud storage service, and the graph storage device 103 can run in a cloud environment, for example, can run on a public cloud, a private cloud, or a hybrid cloud.
  • the image storage device 103 may be an object storage service (English full name: object storage service, English abbreviation: OBS), cloud hard disk, cloud database, etc.
  • the graph computing system shown in FIG. 1 can be provided as a big data engine, and the graph computing device 101 and the graph storage device 103 can be provided as different levels of the big data engine according to logical functions.
  • the graph computing device 101 may be provided as a calculation layer
  • the graph storage device 103 may be provided as a storage layer.
  • the above description is only based on the example that the graph computing device 101 and the graph storage device 103 are two separate devices. In some possible embodiments, the graph computing device 101 and the graph storage device 103 can be integrated Together, that is, the graph computing device 101 may have the functions of computing and storing graphs at the same time.
  • the graph calculation method provided by the embodiments of the present application can be applied to various online and/or offline platforms related to graph calculation, graph analysis, and graph query, and can also be packaged into various applications.
  • the processing logic provided by the following method embodiments can be used as a platform or application built-in graph computing function, so that the platform and/or application can be used in public safety, financial risk control, anti-fraud, social media, root cause analysis, digital asset management , Data traceability and many other fields play a role.
  • the algorithms for graph analysis become more and more abundant; at the same time, the data scale of graphs is also increasing; and cloud platforms usually have multiple at the same time, Even multiple groups of users/tenants are using the platform, and the graphs of each group of users are not the same, often facing a large number of users who need to use multiple analysis algorithms to perform calculations on large-scale graphs, for example, for multiple users under the same tenant
  • the terminals of these users can concurrently request different algorithms to calculate the same graph; for another example, different tenants can request the same algorithm to calculate different graphs through the terminal.
  • the cloud platform can support the function of concurrently executing multiple algorithms on the same graph.
  • the concurrent execution of different algorithms can improve the computing efficiency and computing speed of the cloud platform, thereby improving the cloud platform Performance.
  • different algorithms can share the same graph, so that the number of graph loads will not increase with the increase of the number of algorithms, breaking the input/output (full English name: input/output, English abbreviation: I/O) bottleneck.
  • the terminal of the enterprise may send the social network graph to the graph computing device, and the graph computing device stores the social network graph in the graph storage device.
  • the terminal can send a calculation request to the graph computing device.
  • the calculation request is used to request the use of a web ranking algorithm (hereinafter referred to as algorithm P) and a breadth-first traversal algorithm (hereinafter referred to as algorithm B) to calculate the social network graph, and the graph computing device receives the calculation request Later, the social network graph can be loaded from the graph storage device to the multiple computing units.
  • algorithm P web ranking algorithm
  • algorithm B breadth-first traversal algorithm
  • the multiple computing units can execute Algorithm P and Algorithm B in parallel on the social network graph to obtain the result of Algorithm P and the result of Algorithm B.
  • the result of algorithm P and the result of algorithm B are returned to the terminal of the enterprise.
  • the result of algorithm P is the score of each vertex in the social network graph, which is a floating-point number between 0 and 1, and the score represents the importance of the vertex calculated according to the structure of the social network graph; while the algorithm B's
  • the result is to start from a given root node R, find a pre-order vertex for each vertex on the social network graph, thereby forming a tree starting from the root node R, and associate all reachable vertices.
  • calculation request 1, calculation Request 2 and calculation request 3 respectively indicate that algorithm 1, algorithm 2 and algorithm 3 are used to calculate the same graph.
  • the graph computing device After the graph computing device receives calculation request 1, calculation request 2, and calculation request 3, it can call algorithm 1, algorithm 2, and algorithm 3, and load multiple subgraphs of the graph into the memory of processing unit 1, processing unit 2, and processing unit 3.
  • the bipartite graph between the algorithm and the graph is generated, the bipartite graph between the subgraph and the task is generated according to the bipartite graph, the scheduling scheme between the subgraph and the task is generated according to the bipartite graph, and the scheduling scheme is passed through the processing unit 1.
  • the processing unit 2 and the processing unit 3 execute the tasks of Algorithm 1, Algorithm 2 and Algorithm 3 to execute the scheduling scheme.
  • the graph computing device shown in Figure 3 includes a request cache module, a scheduling module, a distributed task execution module, a first bipartite graph building module, a second bipartite graph building module, a task generation module, a subgraph modal management module, and data Injection module, mode editing module, partition module, sub-graph modal generation module.
  • the modules in FIG. 3 can be used to perform the steps involved in FIG. 2.
  • the first bipartite graph building module can be used to perform the steps of generating a bipartite graph between the algorithm and the graph in Figure 2
  • the second bipartite graph building module can be used to perform the steps between the generated subgraph and the task in Figure 2
  • the scheduling module can be used to execute the steps of generating the scheduling plan in Figure 2.
  • the distributed task execution module can be the processing unit 1, the processing unit 2 and the processing unit 3 in Figure 2, and the distributed task execution module can be used Execute the steps of the scheduling plan in Figure 2.
  • the graph computing device can have two inputs and one output.
  • One of the inputs is raw data.
  • the raw data is used to construct the graph.
  • the raw data is injected into the graph computing device through the data injection module; during the data injection process, the pattern editing module obtains the raw data from the data injection module, according to the mode input by the terminal (English: schema) information, the original data is constructed into a graph.
  • the pattern editing module can construct a graph representing a social network based on a large amount of user data.
  • the partition module is used to partition the constructed graph, and each partition forms a subgraph. After that, each subgraph generates one or more modal subgraphs through the subgraph pattern generation module.
  • the subgraph modal management module can manage the subgraphs of each modal, and each subgraph of each modal It can be persistently stored in the graph storage device for subsequent use when calculating graphs.
  • the mode information is used to indicate the way of constructing the semantics of the graph, that is, the method of defining the data items of the vertices and edges in the graph.
  • the mode information may include vertex declarations, edge declarations, format attributes, mode declarations, etc.
  • the calculation request is also called a query.
  • the calculation request indicates the graph and the algorithm to be executed, for example, contains the name of the algorithm and the name of the graph.
  • the calculation request will be input to the request cache module, and the request cache module will cache the calculation request for batch processing.
  • the first bipartite graph can be generated through the first bipartite graph building module, that is, the bipartite graph between the algorithm and the task.
  • the first bipartite graph will pass the task generation module to convert the algorithm According to the programming model based on its realization, it is decomposed into a series of tasks, such as generating collection tasks, application tasks, and distribution tasks based on the GAS model.
  • the generated task can be associated with the subgraph of the matching modal through the subgraph modal management module to generate a second bipartite graph, that is, the bipartite graph between the task and the subgraph.
  • the scheduling module can schedule the distributed task execution module to execute tasks according to the second bipartite graph. Among them, considering that some algorithms are iterative algorithms and will dynamically generate new tasks with continuous iterations, the scheduling module has an output arrow pointing to the request cache module to dynamically add new iterative tasks to the request cache module.
  • the distributed task execution module is used to execute multiple algorithms.
  • the distributed task execution module can monitor the tasks of the iterative algorithm, save and accumulate the intermediate results of these tasks, and output the final result to the user when the iteration of the iterative algorithm is terminated.
  • the output of the graph calculation device is the calculation result of multiple algorithms, which can be fed back to the user.
  • FIG. 4 shows the process of calculating the graph by the graph computing device using the web ranking algorithm, the BFS algorithm, and the random walk algorithm.
  • the graph computing device provided in FIG. 4 not only includes the modules in FIG. 3, but also includes a data cleaning module, a reading module, a writing module, a visualization module, a reading module, and a scheduling module.
  • a data cleaning module not only includes the modules in FIG. 3, but also includes a data cleaning module, a reading module, a writing module, a visualization module, a reading module, and a scheduling module.
  • FIG. 4 and FIG. 3 please refer to the above description of FIG. 3.
  • the modules and workflows specific to FIG. 4 relative to FIG. 3 are described below.
  • the data cleaning module is used to clean the original data in the data injection module, such as filtering out noise data, etc.
  • the data output by the data cleaning module can be constructed as a graph through the pattern editing module.
  • the reading module is used to read multiple subgraphs of the graph from the graph storage device in the graph calculation process.
  • the writing module is used to write multiple subgraphs of the graph into the graph storage device.
  • the visualization module is used to visualize the results calculated by the calculation unit. For example, it can generate a user interface (English full name: User Interface, English abbreviation: UI) display instructions, and the UI display instructions are used to instruct the terminal to display the user interface including the calculation results.
  • the UI display instruction can be sent to the terminal.
  • the scheduling module is used to schedule tasks in multiple algorithms and multiple subgraphs, so as to perform tasks through the computing unit.
  • FIG. 5 shows another working flowchart of the graph computing device provided in the present application.
  • Fig. 4 shows the process of using multiple algorithms to perform parallel calculation on a single graph
  • Fig. 5 shows the process of using multiple algorithms to perform parallel calculation on multiple graphs.
  • the graph computing device provided in FIG. 5 not only includes the modules in FIG. 3 and FIG. 4, but also includes a load balancing module.
  • the modules and workflows specific to FIG. 5 relative to FIG. 3 and FIG. 4 are described below.
  • the load balancing module is used to load balance multiple computing units. Specifically, as shown in Figure 5, user 1, user 2...user n corresponding to tenant 1 and user 1, user 2...user n corresponding to tenant 2 can simultaneously trigger calculation requests through their respective terminals. The device can simultaneously receive requests from the terminals of the two groups of users corresponding to the two tenants, where n is a positive integer.
  • the load balancing module shown in Figure 5 can determine based on the CPU usage, free memory or other information of multiple computing units that the computing power of multiple computing units may not be sufficient to support tasks generated by processing a large number of computing requests
  • the load balancing module can send an expansion request to the terminal. After receiving the expansion request, the terminal returns an expansion instruction. After the load balancing module expands the computing unit, it can copy the subgraph into multiple instances and load these instances. To the newly added computing unit, so that the newly added computing unit can share the task load.
  • the flow of the graph calculation method is introduced.
  • the method includes steps 601 to 603 performed by the graph calculation device.
  • Step 601 The graph computing device receives at least one computing request.
  • At least one calculation request is used to request that multiple algorithms be used to calculate the graph.
  • each calculation request can be used to request that one or more algorithms be used to calculate the graph.
  • the calculation request may include the identification of the algorithm and the identification of the graph.
  • the identifier of the algorithm is used to indicate the corresponding algorithm, for example, it can be the name of the algorithm, the identity identifier (full English name: identification, English abbreviation: ID), number, etc.
  • ID full English name: identification, English abbreviation: ID
  • the identifier of the graph is used to indicate the corresponding graph, for example, it can be the ID, name, number, etc. of the algorithm.
  • the calculation request may also include a user identification, and the user identification is used to indicate the corresponding user, for example, the user's ID, name, etc.
  • one or more terminals may generate one or more calculation requests and send one or more calculation requests to a graph computing device, and the graph computing device may receive at least one calculation request from one or more terminals.
  • the terminal can run in the terminal environment
  • the graph computing device can run in the cloud environment
  • the terminal and the graph computing device can interact with each other's public network Internet protocol address (English full name: internet protocol address, English abbreviation: IP) address request.
  • IP public network Internet protocol address
  • the graphs requested by the calculation requests of different terminals may be the same or different, and the algorithms requested by the calculation requests of different terminals may be the same or different.
  • the graph computing device can receive calculation requests from multiple terminals at the same time, and execute multiple algorithms on the same graph in parallel by executing steps 602 to 603 below to process calculation requests from multiple terminals.
  • multiple users of multiple tenants may use different algorithms to perform calculations on the same or different graphs at the same time.
  • the terminals of each user can concurrently send calculation requests to the graph computing device, and the graph computing device can receive the calculation requests from multiple terminals at the same time, and execute the following steps 602 to 603 to parallelize each graph in multiple graphs Execute multiple corresponding algorithms to process calculation requests from multiple terminals.
  • the graph computing device may be a computing unit cluster, and the execution subject of step 601 may be one or more computing units in the computing unit cluster.
  • the computing unit cluster may include one or more control plane computing units, and the one or more control plane computing units are used to receive at least one computing request.
  • the computing unit of the control plane may be a master node, a client node, etc. in a computing unit cluster.
  • each computing unit in the computing unit cluster may receive at least one computing request.
  • Step 602 The graph computing device loads multiple subgraphs of the graph to multiple computing units.
  • the graph computing device can obtain the graph ID from the calculation request, determine multiple sub graphs of the graph according to the graph corresponding to the graph ID, and load the multiple sub graphs into the memory of the multiple computing units. After the loading is completed, each calculation The memory of the unit will cache multiple subgraphs.
  • the graph calculation device may load all subgraphs or part of the subgraphs of the graph to the calculation unit. Specifically, if the graph includes a total of N subgraphs, for each calculation unit, the graph calculation device may load all the N subgraphs into the calculation unit; in addition, the graph calculation device may also load M subgraphs into the calculation unit, Where N and M are positive integers, and M is less than N.
  • the graph calculation device may determine the storage capacity of the calculation unit, and if the storage capacity of the calculation unit is sufficient, for example, the storage capacity is greater than a preset threshold, the graph calculation device may load the full amount of subgraphs to the calculation unit; If the storage capacity of the storage capacity of the calculation unit is insufficient, for example, the storage capacity is less than a preset threshold, the graph calculation device may load a part of the subgraph to the calculation unit.
  • the graph computing device can use an average loading method to obtain the ratio between the total number of subgraphs and the total number of calculation units, as the number of subgraphs to be loaded by each calculation unit, and load the number of subgraphs For each calculation unit, the number of subgraphs loaded by such different calculation units can be equal or approximately equal. For example, if the graph includes N subgraphs and the calculation unit includes M, then for each calculation unit, N/M subgraphs can be loaded into the calculation unit, where / means division.
  • multiple subgraphs of the graph may be pre-stored in the graph storage device, and the graph computing device may load the multiple subgraphs from the graph storage device to multiple computing units.
  • the graph computing device may divide the graph into multiple sub graphs in advance, store the multiple sub graphs in the graph storage device, and generate the identification of the graph and the storage of the multiple sub graphs according to the storage location of the multiple sub graphs in the graph storage device. The corresponding relationship between the positions is stored.
  • the storage location of the multiple sub-graphs can be queried from the correspondence relationship according to the map identifier, read the multiple sub-graphs from the storage location in the graph storage device, and transfer the multiple sub-graphs from the graph storage device Load to multiple computing units.
  • the graph computing device may divide the graph into multiple subgraphs in advance, and store the multiple subgraphs obtained by the division into the graph storage device.
  • the graph storage device may store a database for storing sub-graphs of the graph. After the graph computing device divides and obtains the sub-graphs, the sub-graphs can be stored in the database. If a calculation request is received, the The database reads multiple subgraphs of the graph.
  • the graph computing device receives a calculation request, it can determine the graph indicated by the calculation request, divide the graph, and obtain multiple subgraphs.
  • the graph computing device can query whether the graph storage device stores sub graphs of the graph, and if the graph storage device stores multiple sub graphs of the graph, load the multiple sub graphs to the computing unit ; If the graph storage device does not store multiple subgraphs of the graph, the graph indicated by the calculation request is divided into multiple subgraphs, and the multiple subgraphs are loaded into the calculation unit. In addition, multiple subgraphs can be stored in the graph storage device for The next time a calculation request for the graph is received, the subgraph is loaded from the graph storage device.
  • the graph computing device can store new graph subgraphs to the graph storage device multiple times as the number of calculated graphs grows, thereby expanding the data volume of the subgraphs in the graph storage device, and can dynamically Maintain subgraphs of the graph.
  • the graph calculation apparatus may divide the graph into multiple subgraphs after receiving the calculation request. This embodiment does not limit the timing of dividing the graph into subgraphs.
  • the graph computing device may partition the graph, and each partition forms a subgraph.
  • the graph calculation apparatus may divide the graph according to the number of multiple calculation units.
  • the graph calculation device may divide the graph into subgraphs of integer multiples of the number of calculation units. For example, if the total number of multiple computing units is K, the graph computing device may divide the graph into K times subgraphs, and K is a positive integer.
  • the graph can be divided into 10 subgraphs, and if a calculation request is received, 2 subgraphs can be loaded to each virtual machine.
  • the total number of divided subgraphs may be determined according to at least one of the size of the graph and the capabilities of multiple computing units.
  • the total number of divided subgraphs can be positively correlated with the size of the graph.
  • the larger the size of the graph the larger the total number of sub graphs.
  • the total number of subgraphs obtained by the division may be negatively related to the capabilities of multiple computing units.
  • the stronger the capability of the computing unit the smaller the total number of sub-graphs, and the larger the data volume of a single sub-graph.
  • the capability of the computing unit may include the processing capability and storage capability of the computing unit.
  • the processing capability may be represented by indicators such as the clock speed and core number of the computing unit
  • the storage capability may be represented by indicators such as the storage capacity of the computing unit.
  • the total number of sub-graphs can be used as a parameter that can be configured in the graph calculation device, which can be adjusted by the system configuration (English: profiler) tool, or by analyzing historical data of the graph calculation device To adjust, this parameter can be used with various graph partitioning (English: graph partitioning) algorithms.
  • the graph computing device may also divide the graph into multiple subgraphs in a manner of backing up and storing the graph. For example, if the graph is divided into M data blocks during backup storage, each data block of the M data blocks can be regarded as a sub-picture, where M is a positive integer.
  • each subgraph of the graph may include at least one modality
  • the graph computing device may load multiple subgraphs of the at least one modality of the graph to multiple calculation units.
  • the graph calculation device can load the subgraphs of the full modalities of the graph to multiple calculation units. For example, if the subgraph includes W modalities, for each calculation unit, the graph calculation device may load multiple subgraphs of the W modalities of the graph into the calculation unit, where W is a positive integer.
  • the graph includes (2*n) sub-graphs of different modes, the (2 *n)
  • the subgraphs of different modes are the subgraph of the in-edge mode 1, the sub-graph of the out-edge mode 1, the sub-graph of the in-edge mode 2, the sub-graph of the out-edge mode 2...
  • the graph computing device For any computing unit, if the graph computing device wants to load all the subgraphs of the graph into the computing unit, the graph computing device will load (2*n) subgraphs of different modalities into the computing unit; if the graph The calculation device should use the average loading method to load the sub-graphs of the graph to the calculation unit, and assuming that there are m calculation units in total, the graph calculation device will load (2*n/m) sub-graphs of different modalities into the calculation Unit; where n and m are positive integers, and m is less than n, * means multiplication, and / means division.
  • the graph computing device may divide the graph into multiple subgraphs in advance; for each subgraph, generate the subgraph of at least one modality; prestore the multiple subgraphs of the at least one modality in the graph.
  • the graph calculation device can load multiple subgraphs with one less mode from the graph storage device.
  • the graph computing device can generate the subgraph of the in-edge mode and the subgraph of the out-edge mode, and store the subgraphs of both modes in the graph storage device.
  • the solid vertices and hollow vertices in FIG. 7 represent the affiliation (English: ownership) between different vertices in the subgraph, so as to distinguish the vertices contained in the molecular graph itself and the mirror images of the vertices.
  • the graph computing device may obtain the identifiers of multiple algorithms from the calculation request, determine multiple algorithms according to the algorithm corresponding to the algorithm identifiers, and call the multiple algorithms.
  • the process of calling multiple algorithms and step 602 can be executed sequentially. As an example, multiple algorithms may be called first, and then step 602 may be executed; or step 602 may be executed first, and then multiple algorithms may be called.
  • the process of calling multiple algorithms and step 602 can also be executed in parallel, and this embodiment does not limit the process of calling multiple algorithms and the order of step 602.
  • the graph computing device can load multiple sub-graphs to all the computing units rented by the user; in addition, the graph computing device can also select some computing units from all the computing units rented by the user. Load multiple subgraphs into this part of the calculation unit.
  • the graph computing device may temporarily create at least one computing unit in the cluster after receiving the computing request, and load multiple sub-graphs to the created at least one computing unit; in addition, the graph computing device may also, when receiving the computing request, Power on multiple computing units in the dormant state in the cluster, and load multiple subgraphs to the multiple computing units that are powered on; in addition, the graph computing device can also determine that multiple computing units in the cluster are running when receiving a computing request
  • the calculation unit loads multiple subgraphs into multiple calculation units, and this embodiment does not limit which implementation manner is used in implementation.
  • the execution subject of step 602 may be multiple calculation units that need to load subgraphs, that is, step 602 may include: multiple calculation units load multiple subgraphs of the graph.
  • the execution subject of step 602 may also map a calculation unit other than the calculation unit to be loaded into the sub-picture in the calculation device.
  • the graph computing device may be a computing unit cluster.
  • the computing unit cluster includes a computing unit on the control plane and a computing unit on the data plane.
  • the computing unit on the control plane can load multiple subgraphs to the computing unit on the data plane.
  • the computing unit of the control plane can generate control instructions and send control instructions to the computing unit of the data plane.
  • the control instructions are used to instruct to load multiple subgraphs, and the computing unit of the data plane can receive control instructions to load multiple subgraphs of the graph.
  • the computing unit of the data plane may be a slave node in the computing unit cluster.
  • the computing unit of the control plane in the graph computing device may obtain the user ID from the computing request, query the computing unit of the data plane rented by the user ID, and load multiple sub-images to the computing unit of the data plane.
  • Step 603 The graph computing device executes multiple algorithms in parallel through multiple computing units.
  • Multiple computing units may execute multiple algorithms in parallel according to at least one subgraph that has been loaded, that is, multiple algorithms may be executed simultaneously on at least one subgraph.
  • multiple computing units can reuse the same subgraph when executing multiple algorithms.
  • the computing unit can execute the two or more algorithms simultaneously according to the subgraph. For example, if Algorithm 1, Algorithm 2, and Algorithm 3 all need to calculate sub-graph 1, multiple computing units can execute Algorithm 1, Algorithm 2, and Algorithm 3 in parallel according to sub-graph 1.
  • the achieved effects include but are not limited to the following two aspects:
  • any subgraph when a certain algorithm is executed according to the subgraph, other algorithms can be executed at the same time, thus avoiding the delay caused by the need for the algorithm to wait for the execution of other algorithms to end, and shortening the process of multiple algorithms.
  • the overall time of graph calculation improves the overall efficiency of graph calculation by multiple algorithms. For example, when algorithm 1 is executed according to sub-figure 1, algorithm 2 and algorithm 2 can be executed at the same time, avoiding the time delay caused by waiting for the execution of algorithm 1 to end when executing algorithm 2, and also avoiding the time delay caused by waiting for the execution of algorithm 2 when executing algorithm 3. Time delay.
  • the graphs loaded in memory can be reused when multiple algorithms execute graph calculations, instead of loading graphs into memory and loading graphs from memory for each algorithm.
  • the release step it avoids the overhead caused by repeatedly loading and releasing the same graph into the memory, saves the time of accessing the memory, and breaks the IO bottleneck.
  • algorithm 1 and algorithm 2 and algorithm 3 are executed according to subgraph 1
  • the steps of loading subgraph 1 can be executed once for algorithm 1, algorithm 2 and algorithm 3, so as to avoid loading subgraph 1 for algorithm 1 once.
  • the process of loading sub-graph 1 is executed again for algorithm 2, which also avoids the process of loading sub-graph 1 again for algorithm 3, thereby avoiding the delay caused by repeatedly loading the same sub-graph 1.
  • sub-figure 1 can be released again, so as to avoid the release of sub-figure 1 for algorithm 2 when sub-figure 1 has been executed once again.
  • the process also avoids performing the process of releasing sub-graph 1 again for Algorithm 3, thereby avoiding the delay caused by repeatedly releasing the same sub-graph 1.
  • step 603 may include the following steps one to two:
  • Step 1 For each of the multiple algorithms, the graph computing device acquires at least one task in the algorithm.
  • Tasks are also called operators. Tasks are part of the steps in an algorithm.
  • Each algorithm can include one or more tasks. For example, if the algorithm includes Q steps, the task can be P steps in the algorithm, P is greater than or equal to 1 and less than or equal to Q, and P and Q are positive integers.
  • the same algorithm can be divided into different tasks according to different functions. For example, algorithms may include collection tasks, application tasks, and distribution tasks; for another example, algorithms may include local tasks, communication tasks, and synchronization tasks.
  • the graph computing device can decompose the algorithm into tasks with finer granularity by acquiring at least one task in the algorithm, so as to manage and schedule the corresponding algorithm by managing and scheduling the tasks.
  • step one may include any one or a combination of the following manners one to nine.
  • the graph computing device can determine whether the algorithm conforms to the programming model. If the algorithm conforms to the programming model, the following method 1 is executed; if the algorithm does not conform to the programming model, the following method 2 to method 9 are executed.
  • the programming model refers to the specification of programming algorithms.
  • the programming model can be a Pregel model (a vertex-centric programming model), a GAS model, a BSP model, and so on.
  • the algorithm conforms to the programming model, which means that the function name included in the algorithm is consistent with the function name specified in the programming model.
  • the algorithm conforms to the GAS algorithm, the algorithm will include the function named "gather” and the function name "apply"
  • the function name is "scatter"; the algorithm does not conform to the programming model, which means that the function name in the algorithm is inconsistent with the function name specified in the programming model.
  • the function name included in the algorithm is a user-defined function name.
  • Manner 1 The graph computing device divides at least one step corresponding to the same function name in the algorithm into one task according to the function name corresponding to each step in the algorithm.
  • the graph computing device may prestore at least one function name or read at least one function name from an algorithm; the graph computing device may divide at least one step corresponding to the same function name into one task according to the at least one function name.
  • the step corresponding to the function name "gather” in the algorithm can be obtained as a collection task
  • the step corresponding to the function name "apply” in the algorithm can be obtained as an application task
  • the step corresponding to the function name "scatter" in the algorithm can be obtained as a distribution task .
  • Method 2 The graph computing device divides the algorithm into multiple tasks according to the programming model of the algorithm.
  • Mode 2 may include any one or a combination of the following modes (2.1) and (2.2).
  • the graph computing device Based on the GAS model, the graph computing device divides the algorithm into collection tasks, application tasks, and distribution tasks.
  • the graph computing device can divide the processing logic into a collection task according to the processing logic of each step in the algorithm: the step of obtaining data from adjacent vertices of the vertices in the graph and the step of obtaining data on the edges of the graph.
  • the steps to update the data of the vertices in the graph and the steps to synchronize the data of the vertices in the graph to the mirror vertices of the vertices are divided into application tasks.
  • the vertex mirroring vertex step is divided into distribution tasks.
  • the graph computing device can be based on the BSP model to divide the algorithm into local tasks, communication tasks, and synchronization tasks.
  • the graph computing device can divide the processing logic into local tasks according to the processing logic of each step in the algorithm, and divide the processing logic into the steps of interacting with devices other than the local end. It is a communication task; the steps whose processing logic is to wait for the completion of the communication task are divided into synchronization tasks.
  • the graph computing device may divide the multiple algorithms into multiple tasks according to the programming model, and the multiple tasks of different algorithms have the same type. For example, if both Algorithm 1 and Algorithm 2 conform to the GAS model, the graph computing device can divide Algorithm 1 into collection tasks, application tasks, and distribution tasks, and Algorithm 2 can also be divided into collection tasks, application tasks, and distribution tasks.
  • the division of the algorithm into tasks based on the GAS model or the BSP model is only an example.
  • the algorithm can be implemented based on other programming models other than the GAS model or the BSP model. Accordingly, it can be based on other programming models.
  • the programming model divides the algorithm into tasks, and this embodiment does not limit the programming model used in the division algorithm.
  • FIG. 8 shows a schematic diagram of algorithm decomposition and unified management of multiple algorithms.
  • Any algorithm executed by a graph computing device is usually an example of an algorithm, that is, based on a specific algorithm (such as the shortest path algorithm or the subgraph matching algorithm). )
  • Software implementation with a given programming model As shown in Figure 8, the solid box, the dashed box, and the bold box represent different algorithms, and each algorithm is implemented as an example of graph analysis application through a programming model. Take the GAS model as an example.
  • various algorithms can be decomposed into a combination of three tasks, namely collection tasks, application tasks, and distribution tasks. Each task acts on the graph according to a certain rule.
  • the execution process of these three operators forms an iteration of the graph calculation process; among them, the arrow of the distribution task pointing to the collection task indicates that the algorithm is composed of a series of iterations.
  • various algorithms on a graph computing platform can be disassembled into a series of tasks, and these tasks will be allocated to computing units on the cloud or offline. Since each algorithm is broken down into smaller tasks, the execution of multiple algorithms can be mixed together.
  • the data access method of the same type of task is the same. For example, the data access method of the collection task of algorithm 1, algorithm 2 and algorithm 3 is the same, and the data access method of the distribution task of algorithm 1, algorithm 2 and algorithm 3. The way is consistent.
  • the graph computing device can divide the steps with the same execution subject in the algorithm into one task according to the execution subject of each step in the algorithm.
  • the execution subject of a step refers to the hardware or software that executes the step.
  • the execution subject may refer to a computing unit, or a computing unit with computing capability included in the computing unit.
  • the computing unit can be a virtual machine, and the execution subject can be a thread in the virtual machine. If multiple steps in the algorithm are to be executed separately by T threads in the virtual machine, the steps executed by the same thread can be divided into A task.
  • the graph computing device can divide the steps in the algorithm with the same access sequence into one task according to the access sequence of each step in the algorithm to the vertices or edges in the graph.
  • the step from the vertex to the previous vertex in the algorithm can be divided into one task, and the step from the vertex to the subsequent vertex in the algorithm can be divided into another task.
  • the graph computing device can divide the steps with the same vertices or edges visited in the algorithm into one task according to the vertices or edges visited in the graph in each step of the algorithm.
  • step 1 and step 2 need to visit vertex 1
  • step 3 and step 6 need to visit vertex 1001
  • step 5 needs to visit vertex 6001.
  • step 1 and step 2 can be divided into task 1
  • step 3 and step 6 can be divided into task 2
  • step 5 can be divided into task 3.
  • the graph computing device can divide each step in the algorithm into a task.
  • the algorithm can be divided into H tasks, and each task includes 1 step, where H is a positive integer.
  • the graph computing device can divide the steps with the same action in the algorithm into one task according to the actions performed by each step in the algorithm.
  • step 1 executes the action of obtaining data 1 in the graph
  • step 2 executes the action of obtaining data 2 in the graph
  • step 3 executes the action of averaging data 1 and data 2
  • the graph computing device can divide the steps in the algorithm that belong to the same iterative process into one task according to the iterative process to which each step in the algorithm belongs.
  • all steps of the first iteration can be divided into task 1
  • all steps of the second iteration process are divided into task 2
  • all steps of the third iteration process are divided For task 3, and so on.
  • R is a positive integer.
  • the graph computing device can divide the steps in the algorithm that belong to the same judgment branch into one task according to the judgment branch to which each step in the algorithm belongs.
  • the algorithm includes: if the data is greater than the threshold, then perform processing step 1, processing step 2, and processing step 3 on the data, if the data is not greater than the threshold, then perform processing step 6 and processing step 5 on the data, then the processing Step 1, processing step 2, and processing step 3 are divided into task 1, and processing step 6 and processing step 5 are divided into task 2.
  • the graph computing device may execute a part of the algorithm other than the loading step, and the loading step is a step of loading the graph into the memory of the computing unit.
  • the part outside the loading step may be divided into at least one task, and the at least one divided task is executed in parallel through the multiple computing units.
  • the graph computing device may execute the part other than the release step in the algorithm.
  • the release step is a step of releasing the graph from the memory of the computing unit.
  • the part outside the release step may be divided into at least one task, and the at least one divided task is executed in parallel through the multiple computing units.
  • Step 2 The graph computing device executes tasks in multiple algorithms in parallel through multiple computing units.
  • the computing unit can execute tasks in different algorithms in multiple algorithms in parallel on the same subgraph.
  • the graph computing device can allocate tasks in multiple algorithms to the multiple computing units, and each computing unit can execute the tasks allocated by the graph computing device in parallel.
  • the graph computing device can create a task pool. After obtaining the tasks of multiple algorithms, the graph computing device can cache the tasks of multiple algorithms in the task pool; the graph computing device can download the tasks from the task pool. One or more tasks are selected from among them, and one or more tasks are assigned to the multiple computing units; each computing unit can execute the tasks assigned by the graph computing device in parallel. After multiple computing units have performed the assigned tasks, the graph computing device can select the remaining one or more tasks from the task pool, and allocate the remaining one or more tasks to the multiple computing units, each computing unit The tasks allocated by the graph computing device can be executed again in parallel, and so on, until the task pool is empty, that is, the tasks of multiple algorithms have been allocated. Among them, in the process of assigning tasks, each task can be assigned and only assigned to one subgraph of a modal, and each subgraph of each modal can be assigned one or more tasks.
  • the solid line box at the top of Figure 9 includes Algorithm 1, Algorithm 2...Algorithm k, and Figure 1, Figure 2... Figure n, which represents the correspondence between algorithms and graphs and reflects A group of algorithms to be executed in a concurrent execution environment and which graphs need to be processed on each algorithm;
  • the dashed box in the solid line box at the bottom of Figure 9 indicates the correspondence between tasks and subgraphs, which is a comparison of the top of Figure 9
  • the fine-grained expression of the solid line box which reflects a group of tasks to be executed in the concurrent execution environment and which subgraph of each task needs to be processed on which mode; in the process of assigning tasks, it can be regarded as The process of selecting a set of edges from the dashed box.
  • k and n are positive integers.
  • the different algorithms are decomposed into a series of tasks, so that multiple algorithms can be fully mixed together to execute, thereby better supporting the concurrent execution of multiple algorithms.
  • the task granularity is finer than the algorithm granularity, it provides more scheduling space, can make better use of the loaded subgraphs, make it serve more concurrent analysis algorithms, and improve the overall system performance during batch processing.
  • the graph computing device may associate the mode of the subgraph with the task allocation process.
  • step two may include the following steps (2.1) to (2.2):
  • the graph calculation device can load multiple subgraphs of at least one modal of the graph to the multiple calculation units. For example, for each calculation unit in the multiple calculation units, the graph calculation device may load the subgraph of the in-edge mode and the sub-graph of the out-edge mode to the calculation unit.
  • Step (2.2) For any task in multiple algorithms, execute the task according to the subgraph of the target mode.
  • the target mode is a mode that matches the task among at least one mode.
  • the order of the vertices in the subgraph of the target mode can be the same as the order of the vertices in the task.
  • taking the two vertices in the graph as an example the two vertices may be called the first vertex and the second vertex respectively.
  • the task may include searching from the first vertex in the graph to the second vertex in the graph. That is, the first vertex is the vertex visited first in the task, and the second vertex is the vertex visited later in the task.
  • the second vertex in the subgraph of the target mode is arranged before the first vertex, that is, the second vertex is arranged in front of the subgraph, and the first vertex is arranged behind.
  • the update of information on the graph it usually comes from data flowing along edges and calculations performed on vertices.
  • the data flowing along the edge includes two levels: 1) Each vertex obtains the latest data from its previous vertices along the in-edge direction to update the local information of the vertex; 2) Each vertex gets its latest data
  • the local information is propagated to subsequent nodes in the direction of the outgoing edge. Therefore, the subgraph can be expressed as two modes, the in-edge mode represents the in-edge relationship of the sub-graph, and the out-edge mode represents the out-edge relationship of the sub-graph.
  • step (2.2) may include: performing a collection task according to the subgraph of the inbound modal; and performing a distribution task according to the subgraph of the outbound modal.
  • the collection task of multiple algorithms can be executed in parallel according to the same subgraph of the in-edge mode; the distribution task of multiple algorithms can be executed in parallel according to the same subgraph of the out-edge mode.
  • the collection tasks of Algorithm 1, Algorithm 2, and Algorithm 3 can be executed in parallel according to the same subgraph of the in-edge mode.
  • the collection task needs to access the preorder information of each vertex, and the subgraph of the in-edge mode just aggregates the preorder information of each vertex in At the same time, the data locality can be improved; similarly, for the distribution task, the distribution task must provide information to all subsequent nodes, then the distribution task is assigned to the computing unit loaded with the out-of-edge mode for processing. Improve data locality, thereby increasing computational efficiency.
  • a small batch (English: mini-batch) synchronization mechanism can be used to provide information updates for the mirror node.
  • the graph computing device may obtain a scheduling scheme between tasks and subgraphs, and execute tasks of multiple algorithms in parallel according to the scheduling scheme.
  • step 603 may include the following steps A to C:
  • Step A The graph computing device obtains the priorities of multiple algorithms.
  • the priority acquisition method includes but is not limited to a combination of any one or more of the following methods 1 to 2:
  • Manner 1 The graph computing device obtains the priorities of multiple algorithms according to configuration instructions, and the configuration instructions are used to indicate the priorities of the multiple algorithms.
  • the configuration instruction may carry the importance coefficient of each algorithm.
  • the graph computing device may obtain the importance coefficient of each algorithm.
  • the importance coefficient of each algorithm obtain each The priority of the algorithm. Among them, the priority of the algorithm can be positively correlated with the importance coefficient of the algorithm.
  • the importance coefficient indicates the importance of the algorithm. The larger the importance coefficient, the more important the algorithm and the more it needs to be scheduled in time.
  • the importance coefficient can be normalized to control the value range from 0 to 1.
  • Manner 2 The graph computing device obtains the priority of multiple algorithms according to the number of iterations of multiple algorithms.
  • the graph computing device can obtain the number of iterations of multiple algorithms, and perform operations on the number of iterations of multiple algorithms to obtain the priority of the multiple algorithms.
  • the number of iterations of the algorithm is the number of times the algorithm has been used to calculate the graph, that is, the number of times the algorithm has been executed currently.
  • the number of iterations of the algorithm can be less than or equal to the total number of calculations required by the algorithm. In some possible embodiments, if it is expected that the new task can get results as soon as possible, a value negatively related to the number of iterations can be obtained as the priority of the algorithm. Then, because the number of iterations of the new task is small, priority scheduling can be obtained, so as to be scheduled as soon as possible. Executed by the computing unit.
  • the graph calculation device can determine the importance coefficients of multiple algorithms and the number of iterations of multiple algorithms according to any one or a combination of the following formula (1), formula (2), and formula (3) Perform calculations to get the priority of multiple algorithms.
  • a iA,jG ⁇ iA,jG * ⁇ ⁇ A,G> *e -t ⁇ A,G> ;
  • a iA,jG represent the priority of algorithm A to calculate graph G
  • ⁇ A,G> is the importance coefficient of graph G calculated by algorithm A
  • ⁇ A,G> is normalized, ⁇ A,G> is greater than 0 and less than or equal to 1
  • t ⁇ A,G> is the current iteration number of algorithm A to calculate graph G
  • ⁇ iA,jG> is the Dirac Delta function, if calculated
  • the request indicates that algorithm A is used to calculate graph G, and ⁇ iA,jG> takes 1. If the calculation request does not indicate that algorithm A is used to calculate graph G, then ⁇ iA,jG> takes 0, and * means multiplication.
  • the user can control the priority scheduling of new tasks or the priority scheduling of old tasks through configuration instructions. Specifically, if the user desires priority scheduling of new tasks, he can trigger a configuration instruction for instructing the graph computing device to calculate according to formula (1). After receiving the configuration instruction, the graph computing device will obtain the priority of the algorithm according to formula (1). Level, so that the priority of the algorithm is negatively related to the number of iterations of the algorithm. If the user expects that the old tasks can be scheduled preferentially, they can trigger a configuration instruction for instructing the graph computing device to operate according to formula (2). After receiving the configuration instruction, the graph computing device will obtain the priority of the algorithm according to formula (2). The priority of the algorithm is positively correlated with the number of iterations of the algorithm.
  • Step B The graph computing device obtains a scheduling plan according to the priority of each of the multiple algorithms.
  • the scheduling scheme is used to indicate the correspondence between at least one target task and at least one target subgraph.
  • the scheduling scheme may include the identification of at least one target task and the identification of at least one target subgraph.
  • each target task is a task in multiple algorithms
  • the target task is a task scheduled this time among tasks of multiple algorithms
  • the target subgraph is a subgraph scheduled this time for a subgraph in multiple subgraphs.
  • the priority of the algorithm is used to indicate the priority of the algorithm in multiple algorithms. Whenever tasks of multiple algorithms are to be scheduled, the tasks of multiple algorithms with high priority can be scheduled first, and then multiple algorithms with low priority can be scheduled To control the execution end time of different algorithms, so that algorithms with high priority can be executed as soon as possible.
  • the process of generating a scheduling scheme may include the following steps (B.1) to (B.3):
  • Step (B.1) The graph calculating device generates the first bipartite graph.
  • the first bipartite graph is used to indicate the scheduling relationship between the graph and the algorithm, and the first bipartite graph can be regarded as a coarse-grained bipartite graph.
  • the first bipartite graph includes the first vertex, the second vertex and the first edge.
  • the first vertex represents the graph
  • the second vertex represents the algorithm
  • the first edge connects the first vertex and the second vertex
  • the first edge represents the current received
  • the calculation request includes a calculation request for requesting to use the algorithm corresponding to the second vertex to calculate the graph corresponding to the first vertex.
  • the first bipartite graph can be the top bipartite graph in Figure 11, the first vertex is a row of hollow vertices inside the top bipartite graph, and the second vertex is the outer side of the top bipartite graph. A row of solid vertices, the first edge is the realization of the top bipartite graph.
  • the bipartite graph at the top level of Figure 11 there are multiple second vertices connected to the same first vertex, which means that a batch of calculation requests received by the graph computing device includes requests to execute multiple algorithms on the same graph. In this scenario, multiple algorithms can be executed concurrently to improve the calculation speed.
  • Step (B.2) The graph calculation device generates a second bipartite graph based on the first bipartite graph.
  • the second bipartite graph is used to indicate the scheduling relationship between subgraphs and tasks, and the second bipartite graph can be regarded as a fine-grained bipartite graph. It includes a third vertex, a fourth vertex, and a second edge. The third vertex represents the task, the fourth vertex represents the subgraph, and the second edge connects the third vertex and the fourth vertex. Among them, in combination with the modalities of the subgraphs, the second bipartite graph can be used to indicate the scheduling relationship between the subgraphs of any modal and the tasks, and the fourth vertex represents the subgraph of a certain modal. In addition, the second edge can be determined according to the edge of the first bipartite graph.
  • a batch of calculation requests currently received includes a calculation request for calculating graph j using algorithm i
  • the algorithm in the first bipartite graph The first edge between the first vertex corresponding to i and the second vertex corresponding to graph j will be connected.
  • the third vertex corresponding to each task of algorithm i in the second bipartite graph is connected to each child of graph j.
  • the second edge between the corresponding fourth vertices of the graph will be connected.
  • the second bipartite graph may be the bottom bipartite graph in Figure 11
  • the third vertex is a row of solid vertices on the outside of the bottom bipartite graph
  • the fourth vertex is the inner side of the bottom bipartite graph.
  • a row of hollow vertices, the second edge is the solid line in the bottom bipartite graph.
  • the first vertex may be mapped to the fourth vertex according to the correspondence between the graph and the subgraph; the second vertex may be mapped to the third vertex according to the correspondence between the algorithm and the task; Map the first edge to the second edge to get the second bipartite graph.
  • the correspondence between the graph and the subgraph and the correspondence between the algorithm and the task can be represented by the hierarchical relationship between the bipartite graph at the top and the bipartite graph at the bottom.
  • a corresponding relationship between the graph and the subgraph may be represented by an inner dashed line between the bipartite graph on the top layer and the bipartite graph on the bottom layer.
  • the correspondence between the graph A and the subgraph B can be represented by the dotted line between the inner vertex a in the top bipartite graph and the inner vertex b in the bottom bipartite graph.
  • a correspondence between an algorithm and a task can be represented by an outer dashed line between the top bipartite graph and the bottom bipartite graph.
  • the correspondence between algorithm C and task D can be represented by The dotted line between the vertex c and the outer vertex d in the bipartite graph of the bottom layer is indicated.
  • Step (B.3) The graph computing device selects the target edge from the second bipartite graph, and uses the correspondence between the task and the subgraph represented by the target edge as a scheduling plan.
  • the process of obtaining the scheduling plan can be modeled as the process of selecting a set of edges from the bipartite graph at the bottom.
  • the selected edges may be called target edges.
  • the third vertex connected by the target edge is the task to be scheduled this time
  • the fourth vertex connected by the target edge is the subgraph to be scheduled this time.
  • a subgraph can be allocated to one or more Task
  • the same fourth vertex in a group of target edges can be connected to one or more third vertices.
  • the process of selecting a target edge may include the following steps (B.3.1) to (B.3.2).
  • Step (B.3.1) For each edge in the first bipartite graph, the graph computing device obtains the weight of the edge according to the subgraph and task corresponding to the edge.
  • Step (B.3.1) may include a combination of any one or more of the following methods 1 to 2.
  • Method 1 Obtain the importance coefficient corresponding to the edge, and obtain the weight of the edge according to the importance coefficient.
  • the importance coefficient calculated by the algorithm A on the graph G can be obtained, and the importance coefficient can be calculated to obtain the weight, where the weight Positive correlation with the importance coefficient.
  • Method 2 Obtain the number of iterations corresponding to the edge, and obtain the weight of the edge according to the number of iterations.
  • the number of calculations on the graph G using the algorithm A can be obtained, and the number of iterations can be calculated to obtain the weight.
  • mode one and mode two can be executed alternatively or in combination. For example, if method 1 and method 2 are combined, the importance coefficient and iteration number corresponding to the edge can be obtained, and the importance coefficient and iteration number can be calculated to obtain the weight of the edge. For example, any one of the above formula (1) and formula (2) can be used to calculate the importance coefficient and the number of iterations.
  • Step (B.3.2) The graph calculation device selects the target edge from the first bipartite graph according to the weight of each edge in the first bipartite graph, and the weight of the target edge meets the preset condition.
  • the preset condition may include that the sum of weights is the largest among the sum of weights of multiple edges in the second bipartite graph, and the sum of weights is ranked first among the sum of weights of multiple edges in the second bipartite graph. Set the number of bits, the sum of weights to exceed the threshold, etc.
  • the graph computing device may use constraint conditions and adopt a linear programming method to select the target edge. Specifically, the graph computing device can obtain the objective function according to the first bipartite graph; use the linear programming algorithm to calculate the objective function according to the constraint conditions to obtain the solution of the objective function as the objective edge.
  • the objective function is used to select the target edge from the first bipartite graph.
  • the objective function can indicate a set of edges with the largest sum of weights to be solved.
  • the objective function can be represented by the following formula (4).
  • max represents the maximum value
  • Tr represents the trace of the matrix
  • B is the transliteration of the importance matrix A
  • B c+log 2
  • c is a constant.
  • A is the importance coefficient matrix
  • each row in A represents the task obtained after an algorithm decomposition
  • each column in A represents a subgraph of a mode
  • each non-zero element in A represents the task and element corresponding to the element
  • Corresponding subgraphs can be allocated together, and the value of the non-zero element is the importance coefficient of the task performed on the subgraph.
  • X is a premutation matric
  • the function of X is to exchange the columns of B T to maximize the sum of the elements on the diagonal of the exchanged matrix, that is, to find a set of optimal solutions.
  • the corresponding relationship between the task and the subgraph represented by the elements on the diagonal is the selected scheduling plan.
  • Constraints are used to ensure that different edges in the target edges are not disjoint.
  • constraints can be used to constrain that each row of X that is solved by the above formula (4) has and only one 1 and each column has and only A 1.
  • the constraint condition may include the following formula (5), the following formula (6), and the following formula (7), where T represents a transposition operation.
  • the linear programming algorithm may be a subgraph matching algorithm of a bipartite graph, for example, it may be an auction algorithm (English: auction algorithm), a Hungarian path enhancement algorithm (English: Hungarian augmented path algorithm), etc.
  • X in the above formula (4) can be obtained.
  • X is a set of disjoint edges in the second bipartite graph, and X exactly gives a set of scheduling schemes: divide the second bipartite graph The vertex at one end of a certain edge in the graph (representing a task) and the vertex at the other end (representing a subgraph of a certain mode) are scheduled to be executed together.
  • Step C The graph computing device executes at least one target task in parallel through multiple computing units loaded with at least one target subgraph.
  • the graph computing device can determine the target task and target subgraph for this scheduling according to the scheduling plan, determine the computing unit loaded with the target subgraph, and execute the target task in parallel through these computing units loaded with the target subgraph.
  • the graph computing device may repeat the above steps A to C, that is, re-acquire the scheduling plan to obtain the correspondence between the remaining tasks and the subgraph , So as to allocate the remaining tasks until the tasks of multiple algorithms have been executed.
  • Fig. 12 shows an exemplary flow of the graph calculation method.
  • the branch on the left in Fig. 12 can be pre-executed when offline, including the following steps (1) to (4):
  • Step 1 The calculation unit executes the tasks of the first execution process of the iterative algorithm according to multiple subgraphs, and obtains the calculation result of the first execution process.
  • Step 2 The calculation unit judges whether the algorithm converges.
  • the calculation unit may determine that the iterative algorithm has not yet converged according to at least one of the calculation result of the first execution process and the number of iterations. For example, the calculation unit may determine whether the number of iterations has reached the preset number. If the number of iterations has not reached the preset number, the calculation unit determines that the iteration algorithm has not converged; if the number of iterations has reached the preset number, the calculation unit determines that the iteration algorithm has converged. For another example, the calculation unit may determine whether the calculation result of the first execution process meets the preset condition, and if the calculation result of the first execution process does not meet the preset condition, the calculation unit determines that the iterative algorithm has not converged.
  • Step 3 If the iterative algorithm does not converge, the calculation unit executes the task of the second execution process of the iterative algorithm according to the multiple subgraphs and the calculation result of the first execution process, and obtains the calculation result of the second execution process.
  • the calculation unit can re-judgment whether the iterative algorithm has converged. If the iterative algorithm has not yet converged, it executes the tasks of the third execution process of the iterative algorithm according to the multiple subgraphs and the calculation results of the second execution process, and so on, until the iteration
  • the algorithm converges and can output the calculation results of the current execution process, for example, visualize the calculation results of the current execution process, and present the calculation results of the current execution process.
  • the tasks of the algorithms that have not yet converged can be acquired as remaining tasks, thereby dynamically generating new tasks. If the next calculation request is received, the tasks of the algorithm requested in the next calculation request can be obtained, and these new tasks are executed concurrently with the remaining tasks until there are no new calculation requests to be processed.
  • the graph computing device may expand the capacity of the multiple computing units so that more computing units can execute the algorithm, specifically, the expansion
  • the process is detailed in the following steps one to eight.
  • Step 1 The graph computing device sends a capacity expansion request to the terminal.
  • the expansion request is used to request the expansion of multiple computing units.
  • the capacity expansion request may include at least one of the number of computing units to be added and a capacity expansion fee, and the capacity expansion fee is a fee that the user has to pay for the newly added computing unit. For example, if you need to pay n for renting a computing unit within a unit time, but currently you want to add m computing units, the expansion request can include the number of new computing units m and the expansion fee (m*n), where m is Positive integer, n is a positive number.
  • the graph computing device may generate a capacity expansion request and send the capacity expansion request to the terminal when the computing capabilities of multiple computing units do not meet the demand.
  • the calculation request sent by the terminal to the graph computing device may include the expected time length, which is the time that the user expects multiple computing units to complete the graph calculation.
  • the graph computing device can determine whether the multiple computing units can complete the graph calculation within the expected time length.
  • the graph is calculated. If multiple calculation units cannot complete the calculation of the graph within the desired time period, the graph calculation device may generate a capacity expansion request.
  • the graph computing device can monitor the load of each computing unit, and determine whether the load of each computing device exceeds the threshold in real time.
  • the graph computing device can generate an expansion request .
  • multiple computing units can monitor whether their own load conditions exceed the threshold. If their own load conditions exceed the threshold, the multiple computing units send notification messages to the graph computing device, and the graph computing device receives information from one or more computing units. After the notification message, the graph computing device can generate an expansion request.
  • the graph calculation device can determine whether the total data amount that can be calculated by multiple calculation units is greater than the data amount of the graph according to the data amount of the graph and the specifications of each calculation unit. If the total data amount that can be calculated by multiple calculation units is less than Or equal to the data volume of the graph, the graph computing device can generate an expansion request.
  • Step 2 The terminal receives the expansion request of the graph computing device and displays prompt information.
  • the prompt information is used to prompt the user whether to expand the capacity of multiple computing units.
  • the prompt information may also include at least one of the number of computing units to be added and the cost of capacity expansion.
  • Step 3 The terminal receives the confirmation instruction, generates an expansion instruction, and sends the expansion instruction to the graph computing device.
  • the expansion instruction is used to instruct the expansion of multiple computing units.
  • the confirmation instruction is used to instruct to confirm the expansion of multiple computing units, and the confirmation instruction can be triggered by a confirmation operation performed by the user on the terminal.
  • the prompt information displayed by the terminal may include a confirmation option. If the user triggers a click operation on the confirmation option, the terminal receives the confirmation instruction.
  • Step 4 The graph computing device receives the expansion instruction of the terminal.
  • the terminal may send a payment request to the financial server.
  • the payment request includes the expansion fee, the identifier of the user account, and the identifier of the merchant account corresponding to the graph computing device.
  • the financial server After the financial server receives the payment request , Will obtain the user account identifier and the merchant account identifier from the payment request, deduct the expansion fee from the user account, and add the expansion fee to the merchant account; after the deduction is successful, the financial server can send a payment success message to the terminal, and the terminal receives After the payment success message, the expansion instruction is sent to the graph computing device.
  • the graph computing device receives the expansion instruction and executes the following step 5; or the financial server can send the payment success message to the graph computing device, and the graph computing device receives the payment success message and After receiving the expansion instruction, perform the following step five.
  • the graph computing device and the financial server can be integrated.
  • the integrated device has both the function of graph calculation and the function of online transaction.
  • the integrated device can be pre-stored with the identifier of the merchant account. After the terminal receives the confirmation instruction, it can send a payment request to the integrated device. After the integrated device receives the payment request, it will send the payment request from the user account. The expansion fee is deducted and the expansion fee is added to the merchant account; after the deduction is successful, the integrated device can perform the following step five.
  • Step 5 The graph computing device creates at least one computing unit.
  • Step 6 The graph computing device copies at least one subgraph of the graph to obtain an instance of at least one subgraph.
  • the instance of the subgraph is a copy of the subgraph, and the instance of the subgraph can be the same as the subgraph.
  • Step 7 The graph computing device loads at least one instance of the sub graph to the created at least one computing unit.
  • Step 7 is the same as step 602 above, and will not be repeated here.
  • Step 8 The graph computing device executes multiple algorithms in parallel through the created at least one computing unit.
  • Step 8 is the same as the above step 603, and will not be repeated here.
  • the graph computing device may count the number of times each subgraph in the graph is requested by multiple algorithms, determine whether the number of requests for each subgraph reaches the threshold, and copy the subgraphs in the graph whose request times reach the threshold. Obtain instances of the subgraphs whose request times reach the threshold, load the instances of these subgraphs into the expanded target computing unit, and the target computing unit will execute multiple algorithms in parallel on the instances of these subgraphs.
  • the number of requests is the sum of the number of times that multiple algorithms request the subgraph.
  • the graph computing device can perceive the number of requests for each sub-graph.
  • the number of requests for a certain sub-graph exceeds the threshold, it indicates that the demand for the sub-graph is very large, and it is likely to be a hot sub-graph.
  • the subgraph is copied into multiple copies and deployed on different computing units. Multiple computing units process the subgraph, which can improve the processing efficiency of the subgraph. That is, by allowing concurrent computing requests to be diverted to different instances for computing, the concurrency is linearly expanded.
  • multiple subgraphs may be write-protected.
  • the write protection method includes, but is not limited to, any one or a combination of the following methods one to three:
  • Method 1 when any task of any algorithm is written into any subgraph, the computing unit can lock the subgraph. During the execution of the task, other tasks except this task cannot be written The subgraph. When the task of the algorithm has finished writing to the subgraph, the computing unit can release the lock.
  • execution method 1 on the one hand, it can be ensured that the writing steps performed on the subgraphs are executed serially; on the other hand, this method is relatively simple to implement and manage.
  • Method 2 Multi-version concurrency control (English full name: multi-version concurrency control, English abbreviation: MVCC)).
  • the computing unit may cache the full state of each sub-graph in multiple sub-graphs; for example, it may store the current state, historical state, and transition state of the sub-graph.
  • MVCC multi-version concurrency control
  • a target instance can be selected from multiple instances of the subgraph, and the target instance can be regarded as the preferred instance among the multiple instances.
  • the distribution task can be terminated, and it can be resumed after all instances have completed the write operation, thereby achieving stricter consistency.
  • the graph computing device may create a cache space for each task of each algorithm.
  • the graph computing device may cache the intermediate result in the cache space. After the task is executed, the buffer space can be released.
  • the method provided in this embodiment provides a method that can support concurrent execution of graph calculations by multiple algorithms.
  • multiple computing units can execute multiple algorithms in parallel.
  • Multiple algorithms share the same graph, so that multiple algorithms are executed in parallel on the same graph, which saves the time delay caused by waiting for the execution of other algorithms when executing an algorithm, thereby improving the calculation of graphs by multiple algorithms
  • the overall efficiency shortens the overall time of multi-algorithm for graph calculation.
  • the application also provides a graph computing device 1300.
  • the graph computing device 1300 includes a receiving unit 1301, a loading unit 1302, and multiple computing units 1303.
  • the receiving unit 1301 is configured to perform step 601.
  • the loading unit 1302 is configured to perform step 602.
  • Multiple calculation units 1303 are used to perform step 603.
  • the multiple calculation units 1303 include: an acquisition module for performing step one in step 603, for example, it can be used to perform any one of mode one to mode nine in step one in step 603 Item or a combination of multiple items to divide the algorithm into multiple tasks; the execution module is used to execute step two in step 603.
  • the loading unit 1302 is used to load multiple subgraphs of at least one modal to multiple computing units;
  • the execution module is used to execute tasks according to the subgraphs of the target modal;
  • the execution module is used to execute step A to step C in step 603.
  • multiple calculation units 1303 are also used to execute parts other than the loading step in the algorithm.
  • multiple calculation units 1303 are also used to execute parts other than the release step in the algorithm.
  • the device further includes: a dividing unit for dividing the graph; a storage unit for storing multiple subgraphs in the graph storage device; and a loading unit 1302 for storing multiple subgraphs of the graph Load from the graph storage device to multiple computing units.
  • the dividing unit is configured to divide the graph according to the number of multiple calculation units 1303.
  • the device further includes a sending unit for sending an expansion request; a receiving unit for receiving an expansion instruction; a creation unit for creating at least one computing unit; and a copying unit for at least One sub-graph is copied; the loading unit 1302 is also used to load at least one instance of the sub-graph to the created at least one calculation unit; the created at least one calculation unit is used to execute multiple algorithms in parallel.
  • the graph computing device 1300 may be provided to users as a cloud computing service, for example, as a graph engine service. For example, as shown in FIG. 1, the graph computing device 1300 (or a part thereof) is deployed in a cloud environment. After the user triggers a calculation request through a terminal, the graph computing device 1300 is started to execute multiple algorithms on the graph, and the calculation result can be provided to the user.
  • a cloud computing service for example, as a graph engine service.
  • the graph calculation device provided in the embodiment of FIG. 13 performs graph calculation, only the division of the above-mentioned units and modules is used as an example. In actual applications, the above-mentioned functions can be allocated to different units as required. And module completion, that is, the internal structure of the graph computing device is divided into different units and modules to complete all or part of the functions described above.
  • the graph calculation device provided in the above embodiment belongs to the same concept as the graph calculation method embodiment, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
  • the application also provides a graph computing device 1400.
  • the graph computing device 1400 includes multiple computing units 1410.
  • Each computing unit 1410 includes a processor 1411, a memory 1412, a transceiver 1413, and a bus 1414.
  • the processor 1411, the memory 1412, and the transceiver 1413 pass between Bus 1414 for communication.
  • the processor may be a central processing unit (English: central processing unit, abbreviation: CPU).
  • the memory may include a volatile memory (English: volatile memory), such as a random access memory (English: random access memory, abbreviation: RAM).
  • the memory may also include non-volatile memory (English: non-volatile memory), such as read-only memory (English: read-only memory, abbreviation: ROM), flash memory, HDD or SSD.
  • An executable code is stored in the memory, and the processor executes the executable code to execute the aforementioned graph calculation method.
  • the memory may also include an operating system and other software modules required for running processes.
  • the operating system can be LINUX TM , UNIX TM , WINDOWS TM etc.
  • the memory 1412 of each computing unit 1410 of the graph computing device 1400 stores the codes corresponding to the units of the graph computing device 1300, and the processor 1411 executes these codes to realize the functions of the units of the graph computing device 1300, that is, to execute FIG. 6
  • the graph calculation method provided by the embodiment.
  • the graph computing device 1400 can be deployed in a cloud environment. Multiple computing units 1410 in the graph computing device 1400 can form a distributed system, and different computing units 1410 can communicate through a wired or wireless network.
  • the application also provides a graph computing device 1500.
  • the graph computing device 1500 includes multiple computing units 1510, a transceiver 1511, and a bus 1512;
  • Each computing unit 1510 includes a processor 1521 and a memory 1522; wherein, the processor 1521 can be the same as the processor 1411, and the memory 1522 can be the same as the memory 1412, which will not be repeated here.
  • Multiple computing units 1510 in the computing device 1500 of the graph can be deployed in a single machine, and different computing units 1510 can communicate through a bus 1512.
  • the graph computing system includes the graph computing device 1400 shown in the embodiment of FIG. 14, and also includes a graph storage device 1600. Used to store multiple subgraphs of the graph.
  • the graph storage device 1600 may be provided to the user as a cloud storage service, and the user may apply for a certain capacity of storage space in the cloud storage service, and store multiple subgraphs of the graph in the storage space.
  • the graph computing device 1400 When the graph computing device 1400 is running, it loads a plurality of required sub-graphs from the graph storage device 1600 through the communication network.
  • the graph computing device of the embodiment of the present application may be implemented by a general-purpose processor.
  • the general-purpose processor includes a processing circuit and an input interface for internal connection and communication with the processing circuit.
  • the input interface is used to receive at least one calculation request; the processing circuit is used to execute the graph calculation method described above.
  • the general-purpose processor may further include a storage medium, and the storage medium is used to store instructions executed by the processing circuit.
  • the graph computing device of the embodiment of the present application can also be implemented as follows: one or more field programmable gate arrays (English full name: field-programmable gate array, English abbreviation: FPGA), Programmable logic device (English full name: programmable logic device, English abbreviation: PLD), controller, state machine, gate logic, discrete hardware components, any other suitable circuits, or capable of performing various functions described throughout this application Any combination of circuits.
  • field programmable gate arrays English full name: field-programmable gate array, English abbreviation: FPGA
  • Programmable logic device English full name: programmable logic device, English abbreviation: PLD
  • controller state machine
  • gate logic discrete hardware components
  • the present application also provides a computer program product.
  • the computer program product When the computer program product is executed by a graph computing device, the graph computing device executes the foregoing graph computing method.
  • the computer program product may be a software installation package.
  • the computer program product can be downloaded and executed on the graph calculation device.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium It includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program code .
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer program instructions When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.
  • the computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices.
  • Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer instructions can be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, digital subscriber line or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center.
  • the computer-readable storage medium can be any available computer that can be accessed
  • the medium may be a server, data center, and other data storage devices that include one or more available media.
  • the available media may be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, DVDs), or semiconductor media ( For example SSD) etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种图计算方法及装置，能够支持多算法并发执行图计算。通过将图的多个子图加载到多个计算单元，通过多个计算单元，并行执行多个算法，可以在多个算法之间共享同一份图，从而在同一份图上并行执行多个算法，节省了执行某个算法时需要等待其他算法执行结束所造成的时延，从而提高了多算法进行图计算的整体效率，缩短了多算法进行图计算的整体时间。

Description

图计算方法及装置 技术领域
本申请涉及计算机技术领域，尤其涉及一种图计算方法及装置。
背景技术
图是一种基础的数据结构，它包含一系列的节点以及连接节点的边。生活中的许多实体以及实体之间的关系可以通过图上的点和边来直观表达，于是，基于图的分析技术—图计算技术应运而生，应用图计算技术，可以将社交网络、通话网络、用户与产品之间的二分图、论文中作者之间的合作关系网、文章之间的索引关系、金融交易网络、一个地区的无线基站与云端各个服务器之间的交互、手游用户之间的数据交流关系等各种各样的实际应用场景建模为图，通过对图进行计算，来挖掘出图表示的实体所蕴含的规律。
随着图计算技术的推广，经常面临大量用户同时要求对图进行计算的情况，不同用户需要采用相同或不同的算法对图进行计算，因此计算设备会接收到批量化的计算请求，每个计算请求用于请求采用某一种或多种算法对图进行计算。计算设备根据计算请求，会以串行的方式，依次采用每个算法对图进行计算。具体来说，假设计算请求要请求采用N个算法对某个图进行计算，则计算设备会首先将这个图从外存加载到内存，采用算法1对图进行计算，当算法1执行结束，得到算法1的计算结果后，再将图从内存中释放；同理地，再次将图从外存加载到内存，采用算法2对图进行计算，当算法2执行结束，得到算法2的计算结果后，再将图从内存中释放，依次类推，当每个算法依次执行结束后，可以得到N个算法的计算结果。其中，N为正整数。
通过上述示例可以看出，采用串行的方法对图进行计算时，采用一个算法计算结束后，才能采用下一个算法进行计算，导致计算效率低下。
发明内容
本申请提供了一种图计算方法及装置，能够提升图计算的计算效率。
本申请的第一方面提供了一种图计算方法，执行该方法的可以而不限于是图计算装置，例如，图计算装置可以是分布云环境中的多个计算单元。该方法包括：
接收至少一个计算请求，所述至少一个计算请求用于请求采用多个算法对图进行计算；
将所述图的多个子图加载到多个计算单元；
通过所述多个计算单元，并行执行所述多个算法。
以上提供了一种能够支持多算法并发执行图计算的方法，通过将图的多个子图加载到多个计算单元，通过多个计算单元，并行执行多个算法，可以在多个算法之间共享同一份图，从而在同一份图上并行执行多个算法，节省了执行某个算法时需要等待其他算法执行结束所造成的时延，从而提高了多算法进行图计算的整体效率，缩短了 多算法进行图计算的整体时间。
并且，通过在多算法之间共享同一份图，多算法执行图计算时可以复用内存中已载入的图，而无需为每个算法分别执行将图加载到内存以及将图从内存中释放的步骤，也就避免了将同一个图反复地载入内存以及从内存中释放所造成的开销，节省了访问内存的时间，打破了输入/输出(英文全称：input/output，英文简称：IO)瓶颈。
在一种可能的实现方式中，所述通过所述多个计算单元，并行执行所述多个算法，包括：
获取每个算法中的至少一个任务；
通过所述多个计算单元，并行执行所述多个算法中的任务。
在一种可能的实现方式中，所述获取每个算法中的至少一个任务，包括下述至少一项：
根据所述算法中每个步骤对应的函数名称，将所述算法中同一函数名称对应的至少一个步骤划分为一个任务；
基于收集应用分发GAS模型，将所述算法划分为收集任务、应用任务以及分发任务；
基于整体通信同步BSP模型，将所述算法划分为本地任务、通信任务以及同步任务；
根据所述算法中每个步骤的执行主体，将所述算法中执行主体相同的步骤划分为一个任务；
根据所述算法中每个步骤对图中顶点或边的访问顺序，将所述算法中访问顺序相同的步骤划分为一个任务；
根据所述算法中每个步骤在图中访问的顶点或边，将所述算法中访问的顶点或边相同的步骤划分为一个任务；
将所述算法中的每一个步骤划分为一个任务；
根据所述算法中每个步骤执行的动作，将所述算法中动作相同的步骤划分为一个任务；
根据所述算法中每个步骤所属的迭代过程，将所述算法中属于同一次迭代过程的步骤划分为一个任务；
根据所述算法中每个步骤所属的判断分支，将所述算法中属于同一判断分支的步骤划分为一个任务。
以上提供的方法，通过将算法分解为任务，对于处理同一个图的不同算法，能够将具有相同数据访问方式的任务聚集在一起，打破了不同算法之间的隔阂，能够有效地暴露各个任务的数据访问的模式的相似性，从而利用而同一类型的任务对图的存取具有相似的规律性，对任务进行合理的调度和分配，从而合理地利用和调度***资源，提供更高总体性能的服务。尤其是，应用在多用户多算法并发执行，通过对多种算法分解为任务，有助于对多种算法进行统一化管理。
在一种可能的实现方式中，所述将所述图的多个子图加载到多个计算单元，包括：
将至少一种模态的所述多个子图加载到所述多个计算单元；
所述通过所述多个计算单元，并行执行所述多个算法中的任务，包括：
对于所述多个算法的任一任务，通过所述多个计算单元，根据目标模态的子图执行所述任务，所述目标模态为所述至少一种模态中与所述任务匹配的模态。
在一种可能的实现方式中，所述任务包括从所述图中第一顶点搜索至所述图中第二顶点的步骤，所述目标模态的子图中所述第二顶点排在所述第一顶点之前。
以上提供的方法，考虑了不同类型的任务与不同模式的子图之间的亲近关系，实现了图的多模态管理，能够将任务的分配将与子图的模态关联起来，从而将任务分配给装载了合适的子图的计算单元。可以为任务提供相对优化的数据结构，改进在图计算领域中普遍存在的数据局部性有限的问题，从而提高计算任务的执行速度。具体地，由于收集任务要访问每个顶点的前序信息，而入边模态的子图中，正好将每个顶点的前序信息聚合在一起，因此可以提高数据局部性；同理地，对于分发任务来说，分发任务要向所有后序结点提供信息，那么将分发任务分配给装载了出边模态的计算单元进行处理，能提高数据局部性，从而提高计算效率。
在一种可能的实现方式中，所述通过所述多个计算单元，并行执行所述多个算法中的任务，包括：
根据所述多个算法的迭代次数，获取所述多个算法的优先级；
根据每个算法的优先级，获取调度方案，所述调度方案用于指示至少一个目标任务以及至少一个目标子图之间的对应关系，所述目标任务为所述多个算法的任务中本次调度的任务，所述目标子图为所述多个子图中的子图本次调度的子图；
通过加载了所述至少一个目标子图的多个计算单元，并行执行所述至少一个目标任务。
在一种可能的实现方式中，所述通过所述多个计算单元，并行执行所述多个算法中的任务，包括：
根据配置指令，获取所述多个算法的优先级，所述配置指令用于指示所述多个算法的优先级；
根据每个算法的优先级，获取调度方案，所述调度方案用于指示至少一个目标任务以及至少一个目标子图之间的对应关系，所述目标任务为所述多个算法的任务中本次调度的任务，所述目标子图为所述多个子图中的子图本次调度的子图；
通过加载了所述至少一个目标子图的多个计算单元，并行执行所述至少一个目标任务。
以上提供的方法，可以保证算法的优先级的定义方式可以由用户自定义设置，满足用户需求。也即是，有利于***根据不同的调度目标，实施调度，使得***具备良好的可配置能力。
在一种可能的实现方式中，所述通过所述多个计算单元，并行执行所述多个算法，包括下述至少一项：
对于所述多个算法中的任一算法，执行所述算法中的加载步骤之外的部分，所述加载步骤为将所述图加载到计算单元的内存的步骤；
对于所述多个算法中的任一算法，执行所述算法中的释放步骤之外的部分，所述释放步骤为将所述图从计算单元的内存中释放的步骤。
以上提供的方法，对于同一个子图来说，在该子图上执行多个算法的过程中，通 过删除了一个或多个算法中的释放步骤，可以避免执行该一个或多个算法时，要将子图从计算单元的内存释放的过程，也就节省了反复释放同一份子图造成的时间开销以及性能开销。
在一种可能的实现方式中，所述接收至少一个计算请求之前，所述方法还包括：
对所述图进行划分，得到所述多个子图；
将所述多个子图存入图存储装置；
所述将所述图的多个子图加载到至少一个多个计算单元，包括：
将所述图的多个子图从所述图存储装置加载到所述多个计算单元。
以上提供的方法，通过预先将图划分为多个子图并存储多个子图，如果接收到计算请求，可以直接将多个子图加载到该多个计算单元，而无需临时对图划分为子图，也就节省了临时对图划分为子图会造成的时延，可以加快执行多个算法的速度，提升执行多个算法的效率，有助于快速处理完成计算请求。
在一种可能的实现方式中，所述对所述图进行划分，包括：
根据所述多个计算单元的数量，对所述图进行划分。
以上提供的方法，可以支持向多个计算单元平均加载子图，即，将图的多个子图加载到计算单元时，不同计算单元载入的子图的数量相同或近似相同，从而保证每个计算单元根据载入的子图执行算法时，不同计算单元之间负载均衡。
在一种可能的实现方式中，所述方法还包括：
发送扩容请求，所述扩容请求用于请求对所述多个计算单元进行扩容；
接收扩容指令，所述扩容指令用于指示对所述多个计算单元进行扩容；
创建至少一个计算单元；
对所述图的至少一个子图进行复制，得到至少一个子图的实例；
将所述至少一个子图的实例加载到所述创建的至少一个计算单元；
通过所述创建的至少一个计算单元，并行执行所述多个算法。
在一种可能的实现方式中，扩容请求包括要新增的计算单元的数量以及扩容费用中的至少一项。
以上提供的方法，可以在图上并行执行多算法的过程中，自动地进行动态扩容，通过创建更多的计算单元来执行算法，可以提高***整体的计算能力，从而缩短执行多算法的时延。另外，通过将执行算法所需负载的任务量分流到新增的计算单元上，能够实现***的负载均衡。尤其是，应用在用户租用的多个计算单元不足以支持并行执行多算法的场景下，通过自动触发扩容请求，来提示用户租用更多的计算单元以执行算法，可以提高***的自适应能力。
在一种可能的实现方式中，所述对所述图的至少一个子图进行复制，包括：
统计所述图中每个子图被所述多个算法请求的次数；
对所述图中请求次数达到阈值的子图进行复制。
以上提供的方法，考虑到单个图能够支持的访问数量具有物理上限的，如果图的访问数量超过物理上限，很可能在图上无法支持并行执行多个算法，也就对图算法的执行过程造成瓶颈。例如，如果某个子图是热点子图，比如说，该子图是包含了名人信息的子图，同一时刻很可能有大量用户访问该子图以查询名人信息，则该子图可能 无法支持被多个任务同时调度，也就无法并行执行多个算法，导致对图计算的整体速度造成限制。而上述实现方式中，图计算装置能够感知每个子图被请求的数量，如果某个子图的请求次数超过阈值，表明该子图的需求量很大，很可能是热点子图，那么通过触发该子图被复制为多份，分别部署在不同的计算单元上，由多个计算单元对该子图进行处理，能够提高这个子图的处理效率。也即是，通过让并发的计算请求能够分流至不同的实例上来进行计算，从而线性扩展了并发性。
本申请的第二方面提供了一种图计算装置，所述装置用于执行第一方面或第一方面的任意可能的实现方式提供的方法。具体地，该图计算装置包括用于执行第一方面或第一方面的任意可能的实现方式提供的方法的各个单元。
本申请的第三方面提供了一种图计算装置，包括多个计算单元，每个计算单元包括处理器和存储器，所述存储器中存储有至少一条指令，所述指令由所处理器加载并执行以实现第一方面或第一方面的任意可能的实现方式提供的方法。
本申请的第四方面提供了一种图计算***，包括图计算装置和图存储装置；
所述图计算装置包括多个计算单元；
所述图存储装置用于存储图的多个子图；
所述图计算装置用于将所述多个子图从所述图存储装置加载至所述多个计算单元，以执行第一方面或第一方面的任意可能的实现方式提供的方法。
本申请的第五方面提供了一种非瞬态的可读存储介质，所述非瞬态的可读存储介质被图计算装置执行时，所述图计算装置执行前述第一方面或第一方面的任意可能的实现方式中提供的方法。该存储介质中存储了程序。该存储介质的类型包括但不限于易失性存储器，例如随机访问存储器，非易失性存储器，例如快闪存储器、硬盘(hard disk drive，HDD)、固态硬盘(solid state drive，SSD)。
本申请的第六方面提供了一种计算机程序产品，所述计算机程序产品被图计算装置执行时，所述图计算装置执行前述第一方面或第一方面的任意可能的实现方式中提供的方法。该计算机程序产品可以为一个软件安装包，在需要使用前述第一方面或第一方面的任意可能的实现方式中提供的方法的情况下，可以下载该计算机程序产品并在图计算装置上执行该计算机程序产品。
附图说明
为了更清楚地说明本申请实施例的技术方法，下面将对实施例中所需要使用的附图作以简单地介绍。
图1为本申请实施例提供的一种图计算***的架构图；
图2为本申请实施例提供的一种图计算装置的功能架构图；
图3为本申请实施例提供的图计算装置的工作流程图；
图4为本申请实施例提供的图计算装置的另一种工作流程图；
图5为本申请实施例提供的图计算装置的另一种工作流程图；
图6为本申请提供的方法流程示意图；
图7为本申请提供的入边模态的子图以及出边模态的子图的示意图；
图8为本申请提供的算法分解以及多算法统一管理的示意图；
图9为本申请提供的一种多算法的任务调度的示意图；
图10为本申请提供的一种基于子图模态的任务调度的示意图；
图11为本申请提供的一种第一二分图以及第二二分图的示意图；
图12为本申请提供的一种执行迭代算法的流程图；
图13为本申请提供的图计算装置的一种结构示意图；
图14为本申请提供的图计算装置的另一种结构示意图；
图15为本申请提供的图计算装置的另一种结构示意图；
图16为本申请提供的图计算***的一种结构示意图。
具体实施方式
下面结合本申请实施例中的附图，对本申请实施例中的技术方法进行描述。
本申请中的术语“单元”可以通过硬件实现，也可以通过硬件执行相应的软件实现，该硬件或软件具有执行下述方法中相应步骤的功能。例如，接收单元可以由收发器替代，以执行下述方法中的接收步骤，计算单元可以由处理器以及存储器替代，以执行下述方法中的执行步骤。
本申请中的术语“模块”可以为软件模块。
本申请中术语“第一”“第二”等字样用于对作用和功能基本相同的相同项或相似项进行区分，应理解，“第一”、“第二”、“第n”之间不具有逻辑或时序上的依赖关系，也不对数量和执行顺序进行限定。
以下，介绍本申请使用的概念。
图(Graph)：为包括至少一个顶点以及至少一条边的数据结构。在一些场景中，图中的顶点可以映射为实体，图中的边可以映射为实体与实体之间的关系。图可以是有向图或无向图。当然，图还可以包括顶点以及边以外的其他数据，例如顶点的标签以及边的标签等。在一个示例性场景中，应用于好友推荐的场景中，图中的每个顶点可以表示一个用户，图中的每条边可以表示不同用户之间的社交关系，图中每个顶点的数据为用户的画像数据以及用户的行为数据，例如用户的年龄、职业、爱好、学历等。又如，应用于在商品推荐的场景中，图中的每个顶点可以表示一个用户或一个商品，图中的每条边可以表示用户与商品之间的交互关系，例如购买关系、收藏关系等。又如，应用于金融风控的场景中，图中的每个顶点可以表示账号、交易或资金。图中的边可以表示资金的流动关系，例如图中的环路可以表示循环转账。再如，应用于企业网络优化的场景中，图中的每个顶点可以表示一个网元，例如路由器、交换机、终端等，图中的每条边可以表示不同网元之间的连接关系。
算法：包括对图的顶点和/或边的一系列操作，以对图的顶点和/或边进行统计、排序、路径选择等运算。作为示例，算法可以是网页排名算法(也称PageRank算法)，Pixie随机游走算法、广度优先搜索(英文全称：Breadth-First Search,英文简称：BFS)算法、深度优先搜索(英文全称：Depth-First-Search)算法、个性化网页排名算法(也称PersonalRank算法)、k核(英文：k-core)算法、k跳(英文：k-hop)算法、最短路径(英文：shortest paths)算法，全最短路径(英文：all shortest paths)算法、关联路径算法、紧密中心度算法、标签传播算法、基于模块度的社区发现算法 (英文：Louvain算法)、关联预测算法、node2vec算法(一种把网络中的节点映射到欧式空间的算法)、实时推荐算法、共同邻居算法、单源最短路算法、联通分量算法、三角计数算法、聚类系数算法等。
子图(英文：subgraph)：为图的一部分，包括图中的部分顶点以及部分边。子图也可以称为图中的分区(英文：partition)。一个图可以包括多个子图。
收集应用分发(英文全称：gather apply scatter，英文简称：GAS)模型：一种算法的编程规范，按照GAS模型，可以将算法划分为三种任务，分别是收集(英文：gather)任务、应用(英文：apply)任务以及分发(英文：scatter)任务。
收集任务可以包括从图中顶点的相邻顶点获取数据的步骤以及获取图中边的数据的步骤，收集任务可以视为工作顶点从相邻顶点和自身收集数据的过程，例如，收集任务可以为对图中各条边的数据进行求和，又如，收集任务可以是计算某个顶点相邻的顶点的数量。在执行收集任务的过程中，点以及边可以是只读状态。
应用任务可以包括更新图中顶点的数据的步骤以及将图中顶点的数据同步至顶点的镜像顶点的步骤。例如，应用任务可以是各计算顶点计算出来的同一顶点的相邻顶点的数量。应用任务可以是镜像顶点将收集任务的计算结果发送给主顶点，主顶点汇总多个镜像顶点的计算结果，利用汇总结果和上一步的顶点数据，按照业务需求进行进一步的计算，然后更新主顶点的数据并同步到镜像顶点。在执行应用任务的过程中，工作顶点可以是可修改的状态，边可以为不可修改的状态。
分发任务可以包括向图中顶点的相邻顶点发送数据的步骤以及更新图中边的数据的步骤。例如，分发任务可以是工作顶点对数据更新完成之后，更新边上的数据，并通知与工作顶点存在依赖关系的相邻顶点更新状态的过程。在执行分发任务的过程中，工作顶点可以为只读状态，边可以为可写状态。
整体通信同步(英文全称：bulk synchronous parallel computing model，英文简称：BSP)模型：一种算法的编程规范，按照BSP模型，可以将算法划分为三种任务，分别是本地任务、通信任务以及同步任务。
本地任务：也称本地计算(英文：local computation)任务可以包括算法中根据本端的数据进行计算的步骤，例如，本地任务可以包括单个处理单元需要完成的计算过程。
通信(英文：communication)任务可以包括算法中需要不同计算单元进行交互的步骤。例如，通信任务可以包括算法中处理单元A将子图的计算结果发送至处理单元B的步骤。
同步任务可以包括算法中等待该通信任务结束的步骤，例如，同步任务可以包括栅栏同步(barrier synchronization)的过程。
模态(英文：modality)：是指数据的形式，或者说数据的表达方式。对于同一数据来说，该数据的不同模态在形式上有所区别，而包含的实质信息相同。在本申请实施例中，子图可以具有至少一种模态。举例来说，对于某一个子图来说，模态1的该子图包含的顶点和模态2的该子图包含的顶点相同，模态1的该子图包含的边和模态2的该子图包含的边相同，而模态1的该子图包含的顶点的排列顺序和模态2的该子图包含的顶点的排列顺序可以不同。比如说，如果子图包括顶点1、顶点2……顶点 100，模态1的该子图以及模态2的该子图均包含顶点1、顶点2……顶点100，而模态1的该子图中顶点1排在顶点2的前面，而模态2的该子图中顶点1排在顶点2的后面。
入边模态：是指基于入边构建的子图的模态，入边模态用于指示子图中不同顶点之间的入边关系。具体来说，子图中每条边的终点可以排在前，每条边的起点可以排在后。例如，如果子图中包括顶点A、顶点B以及顶点C，这3个顶点之间的关系是：顶点A与顶点B之间存在一条边，且边的方向是顶点A指向顶点B，顶点A与顶点C之间存在另一条边，且边的方向是顶点A指向顶点C，则入边模态的子图中顶点A可以排在前，顶点B和顶点C可以排在后。
出边模态：是指基于出边构建的子图的模态，出边模态的子图可以包括至少一条出边，具体来说，子图中每条边的起点可以排在前，每条边的终点可以排在后。例如，如果子图中包括顶点A、顶点B以及顶点C，这3个顶点之间的关系是：顶点A与顶点B之间存在一条边，且边的方向是顶点A指向顶点B，顶点A与顶点C之间存在另一条边，且边的方向是顶点A指向顶点C，则出边模态的子图中顶点B和顶点C可以排在前，顶点A可以排在后。
以下，示例性介绍本申请的***架构。
图1为本申请实施例提供的一种图计算***的架构图，该图计算***包括图计算装置101以及终端102，终端102用于向图计算装置101发送计算请求，图计算装置101用于接收终端102的计算请求，执行下述方法实施例提供的图计算方法，将计算结果发送至终端102。
如图1所示，图计算装置101可以包括一个或多个计算单元，不同计算单元可以执行下述方法实施例中相同或不同的步骤。在一些可能的实施例中，计算单元可以为物理设备；例如，计算单元可以是物理服务器、中央处理器(英文全称：central processing unit、英文简称：CPU)等。在另一些可能的实施例中，计算单元可以为虚拟化设备；例如，计算单元可以是虚拟机、容器、pod(pod是指Kubernetes中运行、管理、编排容器化应用时容器运行的基本单位)、处理器的核心等等。在另一些可能的实施例中，计算单元可以为软件中的基本单位，例如，计算单元可以为应用、服务、微服务、模块、子模块、类或函数等。其中，图计算装置101中的一个或多个计算单元的规格可以相同，例如一个或多个计算单元可以包括相同数量的处理器以及相同容量的内存，当然一个或多个计算单元的规格也可以存在差异，本实施例对此不做限定。
多个计算单元可以通过单机实现，也可以组成分布式***。在一些可能的实施例中，多个计算单元可以运行在同一物理设备上，不同计算单元可以通过物理设备内部的通讯网络进行通信。例如，多个计算单元可以是一台物理服务器中的多个虚拟机或多个容器；再如，多个计算单元可以是一台物理服务器中通过总线进行通信的多个处理器或者同一个处理器中的多个内核。在另一些可能的实施例中，图计算装置101中的不同计算单元可以运行在不同物理设备上，比如可以运行在不同的地点、不同的计算中心、不同的机房、不同的机架等，图计算装置101中的不同物理设备可以通过网 络交互。
在一些可能的实施例中，图计算装置101可以作为云计算服务向用户提供，例如，可以作为图引擎服务(英文全称：Graph Engine Service，英文简称：GES)向用户提供。图计算装置101可以运行在云环境中，例如可以运行在公有云、私有云或混合云上。作为示例，图计算装置101可以为弹性云服务器(英文全称：elastic cloud server，英文简称：ECS)集群，图计算装置101中的每个计算单元为一个ECS；又如，图计算装置101可以为虚拟机集群，图计算装置101中的每个计算单元为一个在云环境中运行的虚拟机；又如，图计算装置101可以提供为云容器引擎(英文全称：cloud container engine，英文简称：CCE)，图计算装置101中的每个计算单元为一个在云环境中运行的容器；又如，图计算装置101可以提供为云服务平台(英文：cloud service stage)，图计算装置101中的每个计算单元为一个在云环境中运行的应用、服务或微服务。
应理解，图计算装置101运行在云环境中仅是示例，图计算装置101也可以运行在边缘环境中，图计算装置101中的每个计算单元可以为边缘环境中的边缘计算设备；图计算装置101也可以运行在边缘环境中，图计算装置101中的每个计算单元可以为终端环境中的终端设备；本实施例对图计算装置101的运行环境不做限定。另外，图计算装置101的各个计算单元也可以分别运行在不同环境中。例如，图计算装置101可以在云环境、边缘环境、终端环境中的三个，或在其中任意两个环境上运行图计算装置101的部分计算单元。
终端102可以为手机、笔记本、服务器、台式电脑等。终端102可以和图计算装置101通过网络进行交互。
在一些可能的实施例中，该图计算***还可以包括图存储装置103，该图存储装置103用于为图计算装置101提供存储图的服务。其中，该图存储装置103可以通过单机实现，也可以组成分布式***。例如，该图存储装置103可以为分布式存储器。其中，图存储装置103可以通过云存储服务实现，该图存储装置103可以运行在云环境中，例如可以运行在公有云、私有云或混合云上。作为示例，图存储装置103可以为对象存储服务(英文全称：object storage service，英文简称：OBS)、云硬盘、云数据库等。
图1所示的图计算***可以提供为大数据引擎，该图计算装置101以及该图存储装置103可以提供为大数据引擎按照逻辑功能划分的不同层次。例如，该图计算装置101可以提供为计算层，该图存储装置103可以提供为存储层。
需要说明的一点是，上文仅是以图计算装置101以及图存储装置103为两个分离的装置为例进行说明，在一些可能的实施例中，图计算装置101以及图存储装置103可以集成在一起，也即是，图计算装置101可以同时具有对图进行计算以及对图进行存储的功能。
以下，示例性介绍本申请的应用场景。
本申请实施例提供的图计算方法，可以应用于各种与图计算、图分析、图查询等各种相关的线上和/或线下平台，也可以封装为各种不同的应用。其中，下述方法实施 例提供的处理逻辑可以作为平台或应用内置的图计算功能，使得平台和/或应用可以在公共安全、金融风控、反欺诈、社交媒体、根因分析、数字资产管理、数据溯源等很多领域发挥作用。
尤其是，应用在云平台中，由于随着图分析的日益推广，对图进行分析的算法越来越丰富；同时，图的数据规模也不断增加；而云平台上通常会同时有多个，甚至多组用户/租户在使用平台，各组用户的图不尽相同，经常面临大量的用户需要同时使用多种分析算法在大规模的图上进行计算，例如，对于同一租户下的多个用户来说，这些用户的终端可以并发地请求采用不同算法对同一张图进行计算；又如，不同的租户可以通过终端请求使用相同的算法对不同的图进行计算。而通过实施本申请提供的方法，云平台可以支持在同一份图上并发执行多个算法的功能，一方面，不同算法通过并发执行，可以提升云平台的计算效率以及计算速度，从而提升云平台的性能。另一方面，不同算法可以共享同一份图，使得图的装载次数不会随着算法数量的增加而增加，打破了输入/输出(英文全称：input/output，英文简称：I/O)瓶颈。
在一个示例性场景中，可以用于在社交媒体类的企业中进行数据分析和洞察。在此场景中，企业的终端可以向图计算装置发送社交网络图，图计算装置将社交网络图存入图存储装置。终端可以向图计算装置发送计算请求，计算请求用于请求采用网页排名算法(以下简称算法P)和宽度优先遍历算法(以下简称算法B)对社交网络图进行计算，图计算装置接收到计算请求后，可以将社交网络图从图存储装置加载到该多个计算单元，多个计算单元可以在社交网络图上，并行执行算法P和算法B，得到算法P的结果以及算法B的结果，将算法P的结果以及算法B的结果返回给企业的终端。其中算法P的结果为社交网络图中的每个顶点的得分，该得分为0到1之间的浮点数，该得分表示根据社交网络图的结构计算出的顶点的重要性；而算法B的结果则是从给定的根节点R出发，在社交网络图上为每个顶点找到一个前序顶点，从而形成一棵从根节点R出发的树，将所有可达的顶点关联起来。
以下，示例性介绍本申请提供的图计算装置的逻辑功能架构。
参见图2，按照图2中从上至下的顺序来说，如果终端1、终端2以及终端3同时向图计算装置发送了计算请求1、计算请求2以及计算请求3，计算请求1、计算请求2以及计算请求3分别指示采用算法1、算法2以及算法3对同一图进行计算。图计算装置接收到计算请求1、计算请求2以及计算请求3以后，可以调用算法1、算法2以及算法3，将图的多个子图加载到处理单元1、处理单元2以及处理单元3的内存中，生成算法与图之间的二分图，根据该二分图生成子图与任务之间的二分图，根据该二分图生成子图与任务之间的调度方案，按照该调度方案，通过处理单元1、处理单元2以及处理单元3执行算法1、算法2以及算法3的任务，以执行调度方案。
以下，示例性介绍本申请的图计算装置的工作流程。
参见图3，其示出了本申请提供的图计算装置的一种工作流程图。图3所示的图计算装置包括请求缓存模块、调度模块、分布式任务执行模块、第一二分图构建模块、第二二分图构建模块、任务生成模块、子图模态管理模块，数据注入模块、模式编辑模块、分区模块、子图模态生成模块。
其中，图3中的模块可以用于执行图2涉及的步骤。例如，第一二分图构建模块可以用于执行图2中生成算法与图之间的二分图的步骤，第二二分图构建模块可以用于执行图2中生成子图与任务之间的二分图的步骤，调度模块可以用于执行图2中生成调度方案的步骤，分布式任务执行模块可以为图2中的处理单元1、处理单元2以及处理单元3，分布式任务执行模块可以用于执行图2中执行调度方案的步骤。
从图3左侧可见，图计算装置可以具有两个输入和一个输出。其中一个输入为原始数据，该原始数据用于构建图，原始数据经由数据注入模块注入图计算装置中；在数据注入的过程中，模式编辑模块从数据注入模块获取原始数据，根据终端输入的模式(英文：schema)信息，将原始数据构建为图。例如，模式编辑模块可以根据大量的用户数据，构建出表示社交网络的图。分区模块用于对构建的图进行分区，每个分区形成一个子图。之后，每个子图通过子图模式生成模块产生一种或多种模态的子图，子图模态管理模块可以对每个模态的子图进行管理，另外每种模态的每种子图可以被持久化存储至该图存储装置，以供后续对图进行计算的时候调用。
其中，模式信息用于指示构建出图的语义的方式，即定义图中顶点和边的数据项的方法。其中，该模式信息可以包括顶点声明、边声明、格式属性、模式声明等。
而图计算装置的另一个输入则是单个用户或用户群触发的计算请求，该计算请求也称一个查询，该计算请求指示了图和要执行的算法，例如包含算法的名称和图的名称。计算请求会输入请求缓存模块，请求缓存模块会对计算请求进行缓存，以便批量化处理。根据计算请求指示的算法以及图，可以通过第一二分图构建模块，生成第一二分图，即算法与任务之间的二分图，该第一二分图会通过任务生成模块，将算法根据其实现依据的编程模型，分解成一系列的任务，如基于GAS模型生成收集任务、应用任务以及分发任务。生成的任务通过子图模态管理模块，可以与匹配的模态的子图关联，生成第二二分图，即任务和子图之间的二分图。调度模块可以根据第二二分图，调度该分布式任务执行模块执行任务。其中，考虑到一些算法是迭代算法，会随着不断的迭代而动态产生新的任务，调度模块有一个输出箭头指向请求缓存模块，以便动态向请求缓存模块添加新的迭代的任务。分布式任务执行模块用于执行多个算法。另外，分布式任务执行模块可以监控迭代算法的任务，将这些任务的中间结果进行保存和累计，当迭代算法迭代终止后，向用户输出最终的结果。
图计算装置的输出为多个算法的计算结果，可以反馈给用户。
在图3的基础上，参见图4，图4是图3的一个具体实例，图4示出了图计算装置采用网页排名算法、BFS算法以及随机游走算法，对图进行计算的过程。与图3相区别的是，图4提供的图计算装置不仅包含图3中的各个模块，还包含数据清洗模块、读取模块、写入模块、可视化模块、读取模块、调度模块。其中，图4与图3共有的模块以及工作流程还请参见上述对图3的描述，以下描述图4相对于图3来说特有的模块和工作流程。
数据清洗模块用于对数据注入模块中的原始数据进行清洗，例如过滤掉噪声数据等，数据清洗模块输出的数据可以通过模式编辑模块构建为图。读取模块用于在图计算过程中，从图存储装置读取图的多个子图。写入模块用于将图的多个子图写入图存储装置。可视化模块用于对计算单元计算出的结果进行可视化，例如可以生成用户界 面(英文全称：User Interface，英文简称：UI)显示指令，UI显示指令用于指示终端显示包括计算结果的将用户界面，可以将UI显示指令发送至终端。调度模块用于根据多个算法中的任务以及多个子图进行调度，从而通过计算单元执行任务。
在图3以及图4的基础上，参见图5，其示出了本申请提供的图计算装置的另一种工作流程图。与图4区别的是，图4所示的是采用多个算法对单个图进行并行计算的流程，而图5示出了采用多个算法对多个图并行进行计算的流程。并且，与图3和图4相区别是，图5提供的图计算装置不仅包含图3以及图4中的各个模块，图5还包含负载均衡模块。其中，图5与图4以及图3共有的模块以及工作流程，还请参见上述对图3以及图4的描述，以下描述图5相对于图3以及图4来说特有的模块和工作流程。
负载均衡模块用于对多个计算单元进行负载均衡。具体来说，如图5所示，租户1对应的用户1、用户2……用户n，以及租户2对应的用户1、用户2……用户n可以通过各自的终端同时触发计算请求，图计算装置可以同时接收到两个租户对应的两组用户的终端的请求，其中n为正整数。在此场景下，图5所示的负载均衡模块可以根据多个计算单元的CPU使用率、空闲内存或者其他信息，确定多个计算单元的计算能力可能不足以支持处理大量的计算请求产生的任务，为此，负载均衡模块可以向终端发送扩容请求，终端接收到扩容请求后，返回扩容指令，负载均衡模块可以对计算单元进行扩容后，可以将子图复制为多个实例，将这些实例加载到新增的计算单元，以使新增的计算单元分担任务量。
如图6所示，介绍图计算方法的流程，该方法包括由图计算装置执行的步骤601至603。
步骤601、图计算装置接收至少一个计算请求。
至少一个计算请求用于请求采用多个算法对图进行计算。具体来说，每个计算请求可以用于请求采用一个或多个算法对图进行计算。在一些可能的实施例中，计算请求可以包括算法的标识以及图的标识。算法的标识用于指示对应的算法，例如可以为算法的名称、身份标识符(英文全称：identification，英文简称：ID)、编号等。图的标识用于指示对应的图，例如可以为算法的ID、名称、编号等。另外，计算请求还可以包括用户标识，用户标识用于指示对应的用户，例如可以为用户的ID、名称等。
在一些可能的实施例中，一个或多个终端可以生成一个或多个计算请求，向图计算装置发送一个或多个计算请求，图计算装置可以接收一个或多个终端的至少一个计算请求。例如，终端可以运行在终端环境中，图计算装置可以运行在云环境中，终端和图计算装置可以通过彼此的公网互联网协议地址(英文全称：internet protocol address，英文简称：IP)地址交互计算请求。
其中，不同终端的计算请求所请求的图可以相同或不同，不同终端的计算请求所请求的算法可以相同或不同。例如，应用在图4所示的单图多算法的场景中，在同一时刻可以有多个用户要求使用不同的算法在同一个图上进行计算，则多个用户的终端可以并发向图计算装置发送计算请求，图计算装置可以同时接收到多个终端的计算请求，通过执行下述步骤602至步骤603，来在同一个图上并行执行多个算法，以处理 多个终端的计算请求。又如，应用在图5所示的多图多算法的场景中，在同一时刻可以有多个租户的多个用户要使用不同的算法在相同或不同图上进行计算，则多个租户的多个用户的终端可以并发向图计算装置发送计算请求，图计算装置可以同时接收到多个终端的计算请求，通过执行下述步骤602至步骤603，来在多个图中的每个图上并行执行对应的多个算法，以处理多个终端的计算请求。
需要说明的一点是，图计算装置可以为计算单元集群，步骤601的执行主体可以为计算单元集群中的某一个或多个计算单元。在一些可能的实施例中，计算单元集群中可以包括一个或多个控制面的计算单元，这一个或多个控制面的计算单元用于接收至少一个计算请求。例如，该控制面的计算单元可以为计算单元集群中的主节点、客户端节点等。在另一些可能的实施例中，计算单元集群中的每个计算单元均可以接收至少一个计算请求。
步骤602、图计算装置将图的多个子图加载到多个计算单元。
图计算装置可以从计算请求中获取图的标识，根据图的标识对应的图，确定图的多个子图，将多个子图加载到该多个计算单元的内存，则加载完成后，每个计算单元的内存会缓存有多个子图。
图计算装置可以将图的全部的子图或部分子图加载到计算单元。具体地，如果图中总共包括N个子图，对于每个计算单元，图计算装置可以将N个子图均加载到该计算单元；另外，图计算装置也可以将M个子图加载到该计算单元，其中N和M为正整数，且M小于N。
在一些可能的实施例中，图计算装置可以确定计算单元的存储容量，如果计算单元的存储容量足够，例如存储容量大于预设阈值，则图计算装置可以将全量的子图加载到计算单元；如果计算单元的存储容量的存储容量不足，例如存储容量小于预设阈值，则图计算装置可以将部分子图加载到计算单元。
作为示例，图计算装置可以采用平均加载的方式，可以获取子图的总数量与计算单元的总数量之间的比值，作为每个计算单元要加载的子图的数量，将该数量个子图加载到每个计算单元，如此不同计算单元加载的子图数量可以相等或近似相等。例如，如果图中包括N个子图，计算单元包括M个，则对于每个计算单元，可以将N/M个子图加载到该计算单元，其中/表示相除。
在一些可能的实施例中，图的多个子图可以预存在图存储装置中，图计算装置可以将多个子图从图存储装置加载到多个计算单元。例如，图计算装置可以预先将图划分为多个子图，将该多个子图存入图存储装置中，根据多个子图在图存储装置中的存储位置，生成图的标识以及多个子图的存储位置之间的对应关系，存储该对应关系。当接收至少一个计算请求，可以根据图的标识，从对应关系查询到该多个子图的存储位置，从图存储装置中的该存储位置，读取多个子图，将多个子图从图存储装置加载到多个计算单元。
其中，图计算装置可以预先将图划分为多个子图，将划分得到的多个子图存入图存储装置中。作为示例，图存储装置可以存储有数据库，该数据库用于存储图的子图，当图计算装置划分得到子图后，可以将子图存入该数据库中，如果接收到计算请求，则从该数据库读取图的多个子图。
通过预先将图划分为多个子图并存储多个子图，如果接收到计算请求，可以直接将多个子图加载到该多个计算单元，而无需临时对图划分为子图，也就节省了临时对图划分为子图会造成的时延，可以加快执行多个算法的速度，提升执行多个算法的效率，有助于快速处理完成终端的计算请求。
当然，预先将图划分为子图仅是示例，在另一些可能的实施例中，如果图计算装置接收到计算请求，可以确定计算请求指示的图，对图进行划分，得到多个子图。
可选地，如果图计算装置接收到计算请求，图计算装置可以查询图存储装置是否存储有图的子图，如果图存储装置存储有图的多个子图，则将多个子图加载到计算单元；如果图存储装置未存储有图的多个子图，则将计算请求指示的图划分为多个子图，将多个子图加载到计算单元，另外，可以将多个子图存入图存储装置，以便下次接收到对该图的计算请求时，从图存储装置加载子图。
通过这种实现方式，图计算装置可以随着计算的图的数量的增长，多次向图存储装置存入新的图的子图，从而扩大图存储装置中子图的数据量，可以动态地维护图的子图。
在另一些可能的实施例中，图计算装置可以在接收到计算请求后，再将图划分为多个子图，本实施例对将图划分为子图的时机不做限定。
关于对图划分为子图的过程，图计算装置可以对图进行分区(partition)，每个分区形成一个子图。在一些可能的实施例中，图计算装置可以根据多个计算单元的数量，对图进行划分。作为示例，图计算装置可以将图划分为计算单元的数量的整数倍的子图。比如说，如果多个计算单元的总数量为K个，图计算装置可以将图划分为K倍个子图，K为正整数。
通过这种实现方式，可以支持向多个计算单元平均加载子图，保证将子图加载到计算单元时，不同计算单元载入的子图的数量相同或近似相同。作为示例，比如用户租用了5个虚拟机，可以将图划分为10个子图，如果接收到计算请求，可以向每个虚拟机加载2个子图。
作为示例，划分得到的子图的总数量可以根据图的大小以及多个计算单元的能力中的至少一项确定。划分得到的子图的总数量可以与图的大小正相关。图的大小越大，则子图的总数量可以越大。划分得到的子图的总数量可以与多个计算单元的能力负相关。计算单元的能力越强，则子图的总数量可以越小，则单个子图的数据量越大。其中，计算单元的能力可以包括计算单元的处理能力以及存储能力，该处理能力可以通过计算单元的主频、核数等指标表示，该存储能力可以通过计算单元的存储容量等指标表示。
在一些可能的实施例中，子图的总数量可以作为能够在图计算装置配置的参数，该参数可通过***的配置(英文：profiler)工具调整，或通过对图计算装置的历史数据进行分析来调整，该参数可以配合各类图划分(英文：graph partitioning)算法使用。
在另一些可能的实施例中，图计算装置也可以按照对图备份存储的方式，将图划分为多个子图。例如，如果备份存储时，将图划分为M个数据块，可以将M个数据块中的每个数据块作为一个子图，其中M为正整数。
在一些可能的实施例中，图的每个子图可以包括至少一种模态，图计算装置可以将图的至少一种模态的多个子图加载到多个计算单元。其中，图计算装置可以将图的全量的模态的子图加载到多个计算单元。例如，如果子图包括W种模态，对于每个计算单元，图计算装置可以将图的W种模态的多个子图分别加载到计算单元，其中W为正整数。
以子图包括入边模态和出边模态这2种模态为例，如果图中总共包括n个子图，则图总共包括(2*n)个不同模态的子图，该(2*n)个不同模态的子图分别是入边模态的子图1、出边模态的子图1、入边模态的子图2、出边模态的子图2……入边模态的子图n、出边模态的子图n。对于任一计算单元来说，如果图计算装置要将图的全量的子图加载到计算单元，则图计算装置会将(2*n)个不同模态的子图加载到计算单元；如果图计算装置要采用平均加载的方式，将图的子图加载到计算单元，并假设计算单元共计m个，则图计算装置会将(2*n/m)个不同模态的子图加载到计算单元；其中，n和m为正整数，且m小于n，*表示相乘，/表示相除。
在一些可能的实施例中，图计算装置可以预先将图划分为多个子图；对于每个子图，生成至少一种模态的该子图；将至少一种模态的多个子图预存在图存储装置中；如果接收到计算请求，图计算装置可以从图存储装置加载到少一种模态的多个子图。例如，参见图7，对于图中的每个子图，图计算装置可以生成入边模态的子图以及出边模态的子图，将两种模态的该子图均存入图存储装置中。其中，图7中的实心顶点与空心顶点表示子图中不同顶点之间的从属关系(英文：ownership)，以区分子图本身包含的顶点以及顶点的镜像。
在一些可能的实施例中，图计算装置可以从计算请求中获取多个算法的标识，根据算法的标识对应的算法，确定多个算法，调用该多个算法。其中，调用多个算法的过程与步骤602可以顺序执行。作为示例，可以先调用多个算法，再执行步骤602；也可以先执行步骤602，再调用多个算法。当然，调用多个算法的过程与步骤602也可以并行执行，本实施例对调用多个算法的过程与步骤602的顺序不做限定。
在一个示例性场景中，应用于云计算领域，图计算装置可以将多个子图加载到用户租用的全部计算单元；另外，图计算装置也可以从用户租用的全部计算单元中选择部分计算单元，将多个子图加载到该部分计算单元。
其中，图计算装置可以在接收到计算请求后，在集群中临时创建至少一个计算单元，将多个子图加载到创建的至少一个计算单元；另外，图计算装置也可以在接收到计算请求时，上电集群中处于休眠状态的多个计算单元，将多个子图加载到上电的多个计算单元；另外，图计算装置也可以在接收到计算请求时，确定集群中处于运行态的多个计算单元，将多个子图加载到多个计算单元，本实施例对实施中采用哪种实现方式不做限定。
需要说明的一点是，步骤602的执行主体可以就是要加载子图的多个计算单元，即，步骤602可以包括：多个计算单元加载图的多个子图。步骤602的执行主体也可以图计算装置中要加载子图的计算单元以外的计算单元。例如，图计算装置可以为计算单元集群，计算单元集群包括控制面的计算单元以及数据面的计算单元，控制面的计算单元可以将多个子图加载到数据面的计算单元。例如，控制面的计算单元可以生 成控制指令，向数据面的计算单元发送控制指令，控制指令用于指示加载多个子图，数据面的计算单元可以接收控制指令，加载图的多个子图。其中，该数据面的计算单元可以为计算单元集群中的从节点。作为示例，图计算装置中控制面的计算单元可以从计算请求中获取用户标识，查询该用户标识租用的数据面的计算单元，将多个子图加载到该数据面的计算单元。
步骤603、图计算装置通过多个计算单元，并行执行多个算法。
多个计算单元可以根据已加载的至少一个子图，并行执行多个算法，也即是，可以在至少一个子图上同时执行多个算法。其中，多个计算单元在执行多个算法时可以复用同一个子图。具体来说，如果任两个或两个以上的算法的输入是同一个子图，则计算单元可以根据该子图，同时执行该两个或两个以上的算法。例如，如果算法1、算法2以及算法3均需要对子图1进行计算，则多个计算单元可以根据子图1，并行执行算法1、算法2以及算法3。
通过在至少一个子图上并行执行多个算法，达到的效果包括而不限于下述两个方面：
一方面，对于任一个子图来说，当根据该子图执行某个算法时，可以同时执行其他算法，从而避免了该算法需要等待其他算法执行结束所造成的时延，缩短了多算法进行图计算的整体时间，提高了多算法进行图计算的整体效率。例如，在根据子图1执行算法1时，可以同时执行算法2以及算法2，避免了执行算法2时等待算法1执行结束造成的时延，也避免了执行算法3时等待算法2执行结束造成的时延。
另一方面，通过在多算法之间共享同一份图，多算法执行图计算时可以复用内存中已载入的图，而无需为每个算法分别执行将图加载到内存以及将图从内存中释放的步骤，也就避免了将同一个图反复地载入内存以及从内存中释放所造成的开销，节省了访问内存的时间，打破了IO瓶颈。例如，在根据子图1执行算法1以及算法2以及算法3时，可以为算法1、算法2以及算法3执行一次加载子图1的步骤，避免在已经为算法1执行一遍加载子图1的过程的情况下，再次为算法2执行一遍加载子图1的过程，也避免了再次为算法3执行一遍加载子图1的过程，从而避免反复加载同一个子图1造成的时延。另外，可以当算法1、算法2以及算法3均执行结束后，再释放子图1，避免在已经为算法1执行一遍释放子图1的情况下，再次为算法2执行一遍释放子图1的过程，也避免了再次为算法3执行一遍释放子图1的过程，从而避免反复释放同一个子图1造成的时延。
在一些可能的实施例中，步骤603可以包括下述步骤一至步骤二：
步骤一、对于多个算法中的每个算法，图计算装置获取算法中的至少一个任务。
任务也称算子，任务为算法中的部分步骤，每个算法可以包括一个或多个任务。例如，如果算法包括Q个步骤，任务可以是算法中的P个步骤，P大于或等于1且小于或等于Q，P和Q为正整数。同一算法可以按照不同的功能，划分为不同任务。例如，算法可以包括收集任务、应用任务以及分发任务；又如，算法可以包括本地任务、通信任务以及同步任务。
图计算装置通过获取算法中的至少一个任务，可以将算法分解为粒度更细的任务，以便通过对任务进行管理和调度，来对相应的算法进行管理和调度。
在一些可能的实施例中，步骤一可以包括下述方式一至方式九中的任一项或多项的组合。可选地，图计算装置可以判断算法是否符合编程模型，如果算法符合编程模型，则执行下述方式一，如果算法不符合编程模型，则执行下述方式二至至方式九。其中，编程模型是指编写算法的规范，例如，编程模型可以为Pregel模型(一种以顶点为中心的编程模型)、GAS模型、BSP模型等。算法符合编程模型，是指算法包含的函数名称和编程模型中规定的函数名称一致，例如，如果算法符合GAS算法，则算法中会包含函数名称为“gather”的函数、函数名称为“apply”的函数，函数名称为“scatter”的函数；算法不符合编程模型，是指算法中的函数名称和编程模型中规定的函数名称不一致，例如算法包含的函数名称是用户自定义的函数名称。
方式一、图计算装置根据算法中每个步骤对应的函数名称，将算法中同一函数名称对应的至少一个步骤划分为一个任务。
作为示例，图计算装置可以预存至少一个函数名称，或者从算法中读取至少一个函数名称；图计算装置可以根据至少一个函数名称，将同一函数名称对应的至少一个步骤划分为一个任务。例如，可以获取算法中函数名称“gather”对应的步骤，作为收集任务，获取算法中函数名称“apply”对应的步骤，作为应用任务，获取算法中函数名称“scatter”对应的步骤，作为分发任务。
方式二、图计算装置根据算法的编程模型，将算法划分为多个任务。
方式二可以包括下述方式(2.1)方式(2.2)中的任一项或多项的组合。
方式(2.1)图计算装置基于GAS模型，将算法划分为收集任务、应用任务以及分发任务。
作为示例，图计算装置可以根据算法中每个步骤的处理逻辑，将处理逻辑为从图中顶点的相邻顶点获取数据的步骤以及获取图中边的数据的步骤划分为收集任务，将处理逻辑为更新图中顶点的数据的步骤以及将图中顶点的数据同步至顶点的镜像顶点的步骤划分为应用任务，将处理逻辑为更新图中顶点的数据的步骤以及将图中顶点的数据同步至顶点的镜像顶点的步骤划分为分发任务。
方式(2.2)图计算装置可以基于BSP模型，将算法划分为本地任务、通信任务以及同步任务。
作为示例，图计算装置可以根据算法中每个步骤的处理逻辑，将处理逻辑为根据本端的数据进行计算的步骤划分为本地任务；将处理逻辑为与本端以外的其他设备进行交互的步骤划分为通信任务；将处理逻辑为等待该通信任务结束的步骤划分为同步任务。
在一些可能的实施例中，对于采用同一编程模型的多个算法来说，图计算装置可以根据该编程模型，将多个算法划分为多个任务，则不同算法的多个任务的类型相同。例如，如果算法1以及算法2均符合GAS模型，则图计算装置可以将算法1划分为收集任务、应用任务以及分发任务，将算法2也划分为收集任务、应用任务以及分发任务。
需要说明的一点是，基于GAS模型或BSP模型将算法划分为任务仅是示例，在一些可能的实施例中，算法可以基于GAS模型或BSP模型以外的其他编程模型实现，相应地，可以基于其他编程模型将算法划分为任务，本实施例对划分算法所采用的编程 模型不做限定。
示例性地，图8示出了算法分解以及多算法统一管理的示意图，图计算装置执行的任一算法通常为算法的一个实例，即基于一个具体的算法(如最短路径算法或子图匹配算法)与一个给定的编程模型的软件实现。如图8所示，实线方框、虚线方框以及加粗方框分别代表不同的算法，每个算法通过一个编程模型实现成为一个图分析应用的实例。以GAS模型为例，根据这种编程模型，各类算法都能够被分解成三种任务的组合，即收集任务、应用任务以及分发任务，每个任务按一定规律作用在图上。这三个算子的执行过程，形成图计算过程的一次迭代；其中，分发任务指向收集任务的箭头表示算法是由一系列迭代组成的。由此可见，可以把在一个图计算平台上各式各样的算法都拆解成一系列的任务，这些任务将会被分配到云上或者线下的计算单元上去。由于每个算法被分解成更小的任务，多个算法的执行可以混在一起进行。对于不同算法，同一类型的任务的数据访问方式是一致的，例如算法1、算法2以及算法3的收集任务的数据访问方式是一致的，算法1、算法2以及算法3的分发任务的数据访问方式是一致的。
通过将算法分解为任务，对于处理同一个图的不同算法，能够将具有相同数据访问方式的任务聚集在一起，打破了不同算法之间的隔阂，能够有效地暴露各个任务的数据访问的模式的相似性，从而利用而同一类型的任务对图的存取具有相似的规律性，对任务进行合理的调度和分配，从而合理地利用和调度***资源，提供更高总体性能的服务。尤其是，应用在多用户多算法并发执行，通过对多种算法分解为任务，有助于对多种算法进行统一化管理。
方式三、图计算装置可以根据算法中每个步骤的执行主体，将算法中执行主体相同的步骤划分为一个任务。
步骤的执行主体是指执行步骤的硬件或软件，在本实施例中，执行主体可以指计算单元，也可以指计算单元包含的具有计算能力的计算单位。举例来说，计算单元可以是虚拟机，执行主体可以是虚拟机中的线程，如果算法中的多个步骤要通过虚拟机中的T个线程分别执行，可以将同一个线程执行的步骤划分为一个任务。
方式四、图计算装置可以根据算法中每个步骤对图中顶点或边的访问顺序，将算法中访问顺序相同的步骤划分为一个任务。
例如，可以将算法中从顶点访问至前续顶点的步骤划分为一个任务，将算法中从顶点访问至后继顶点的步骤划分为另一个任务。
方式五、图计算装置可以根据算法中每个步骤在图中访问的顶点或边，将算法中访问的顶点或边相同的步骤划分为一个任务。
例如，如果算法中包含5个步骤，步骤1以及步骤2要访问顶点1、顶点2至顶点1000，步骤3以及步骤6要访问顶点1001、顶点1002至顶点6000，步骤5要访问顶点6001、顶点6002至顶点7000，则可以将步骤1以及步骤2划分为任务1，将步骤3以及步骤6划分为任务2，将步骤5划分为任务3。
方式六、图计算装置可以将算法中的每一个步骤划分为一个任务。
例如，如果算法中包含H个步骤，可以将算法划分为H个任务，每个任务包括1个步骤，其中H为正整数。
方式七、图计算装置可以根据算法中每个步骤执行的动作，将算法中动作相同的步骤划分为一个任务。
例如，如果算法中包含3个步骤，步骤1执行获取图中数据1的动作，步骤2要执行获取图中数据2的动作，步骤3要执行数据1和数据2求取平均值的动作，则可以将步骤1以及步骤2划分为任务1，将步骤3划分为任务2。
方式八、图计算装置可以根据算法中每个步骤所属的迭代过程，将算法中属于同一次迭代过程的步骤划分为一个任务。
例如，如果算法中包含R次迭代过程，可以将第1次迭代过程的所有步骤划分为任务1，将第2次迭代过程的所有步骤划分为任务2，将第3次迭代过程的所有步骤划分为任务3，依次类推。其中R为正整数。
方式九、图计算装置可以根据算法中每个步骤所属的判断分支，将算法中属于同一判断分支的步骤划分为一个任务。
例如，如果算法中包括：如果数据大于阈值，则对数据执行处理步骤1、处理步骤2以及处理步骤3，如果数据不大于阈值，则对数据执行处理步骤6以及处理步骤5，则可以将处理步骤1、处理步骤2以及处理步骤3划分为任务1，将处理步骤6以及处理步骤5划分为任务2。
在一些可能的实施例中，对于多个算法中的任一算法，图计算装置可以执行该算法中的加载步骤之外的部分，该加载步骤为将该图加载到计算单元的内存的步骤。例如，可以将加载步骤之外的部分划分为至少一个任务，通过该多个计算单元，并行执行划分出的至少一个任务。
通过这种实现方式，对于同一个子图来说，在该子图上执行多个算法的过程中，通过避免执行该一个或多个算法时，要将子图加载到计算单元的内存的过程，也就节省了反复加载同一份子图造成的时间开销以及性能开销。
在一些可能的实施例中，对于多个算法中的任一算法，图计算装置可以执行该算法中的释放步骤之外的部分。其中，释放步骤为将图从计算单元的内存中释放的步骤。例如，可以将释放步骤之外的部分划分为至少一个任务，通过该多个计算单元，并行执行划分出的至少一个任务。
通过这种实现方式，对于同一个子图来说，在该子图上执行多个算法的过程中，通过避免执行该一个或多个算法时，要将子图从计算单元的内存释放的过程，也就节省了反复释放同一份子图造成的时间开销以及性能开销。
步骤二、图计算装置通过多个计算单元，并行执行多个算法中的任务。
作为示例，对于任一计算单元来说，该计算单元可以在同一个子图上，并行执行多个算法中不同算法中的任务。其中，图计算装置可以将多个算法中的任务分配给该多个计算单元，每个计算单元可以并行执行图计算装置分配的任务。
在一些可能的实施例中，图计算装置可以创建任务池，当获取到多个算法的任务后，图计算装置可以将多个算法的任务缓存在该任务池中；图计算装置可以从任务池中选取一个或多个任务，将一个或多个任务分配给该多个计算单元；每个计算单元可以并行执行图计算装置分配的任务。当多个计算单元对分配的任务执行结束后，图计算装置可以从任务池中选取剩余的一个或多个任务，将剩余的一个或多个任务分配给 该多个计算单元，每个计算单元可以再次并行执行图计算装置分配的任务，依次类推，直至任务池为空，即多个算法的任务均已分配为止。其中，在分配任务的过程中，每个任务可以被分配且仅被分配到一个模态的一个子图上，而每个模态的每个子图可以被分配一个或多个任务。
示例性地，参见图9，图9上方的实线框包括算法1、算法2……算法k，以及图1、图2……图n，它表示算法和图之间的对应关系，反映了并发执行环境中一组待执行的算法和每个算法上需要在哪些图上进行处理；图9下方的实线框中的虚线框表示任务和子图之间的对应关系，是对图9上方的实线框的细粒度的表达，它反映了并发执行环境中一组待执行的任务和每个任务上需要在哪个模态的哪个子图上进行处理；在分配任务的过程中，可以视为从虚线框中选择出一组边的过程。其中，k和n为正整数。
通过上述步骤一以及步骤二，将不同的算法分解为了一系列的任务，让多个算法可以充分地混在一起执行，从而更好地支持了多个算法的并发执行。同时，由于任务的粒度比算法的粒度更细，提供了更多的调度空间，能够更好的利用已经加载的子图，使其服务更多并发分析的算法，提升批量处理时的***总体性能，
在一些可能的实施例中，图计算装置可以将子图的模态与任务的分配过程关联起来。具体来说，步骤二可以包括下述步骤(2.1)至步骤(2.2)：
步骤(2.1)图计算装置可以将图的至少一种模态的多个子图加载到该多个计算单元。例如，对于多个计算单元中的每个计算单元，图计算装置可以将入边模态的子图和出边模态的子图加载到该计算单元。
步骤(2.2)对于多个算法中的任一任务，根据目标模态的子图执行该任务。
目标模态为至少一种模态中与任务匹配的模态。作为示例，目标模态的子图中顶点的排列顺序可以任务中对顶点的搜索顺序一致。在一些可能的实施例中，以图中的两个顶点为例，不妨称这两个顶点分别为第一顶点以及第二顶点，任务可以包括从图中第一顶点搜索至图中第二顶点的步骤，即，第一顶点是任务中先访问的顶点，第二顶点是任务中后访问的顶点。目标模态的子图中第二顶点排在第一顶点之前，即，在子图中第二顶点排在前，第一顶点排在后。
在一些可能的实施例中，考虑到图上信息的更新，通常来自沿着边流动的数据以及在顶点上进行的计算。而沿着边流动的数据包括两个层面：1)各个顶点沿着入边方向从其前续顶点中获取最新的数据，用于更新该顶点的本地信息；2)每个顶点将其最新的本地信息沿着出边的方向传播到后继结点。因此，可以将子图表达为两个模态，入边模态表示子图的入边关系，出边模态表示子图的出边关系。
相应地，步骤(2.2)可以包括：根据入边模态的子图，执行收集任务；根据出边模态的子图，执行分发任务。其中，可以根据入边模态的同一子图，并行执行多个算法的收集任务；根据出边模态的同一子图，并行执行多个算法的分发任务。例如，参见图10，可以根据入边模态的同一子图，并行执行算法1、算法2以及算法3的收集任务。
通过上述步骤(2.1)至步骤(2.2)，考虑了不同类型的任务与不同模式的子图之间的亲近关系(英文：affinity)，实现了图的多模态管理，能够将任务的分配将与子 图的模态关联起来，从而将任务分配给装载了合适的子图的计算单元。可以为任务提供相对优化的数据结构，改进在图计算领域中普遍存在的数据局部性(英文：data locality)有限的问题，从而提高计算任务的执行速度。例如，基于GAS模型构建的算法中，算法分解出的任务中，收集任务要访问每个顶点的前序信息，而入边模态的子图中，正好将每个顶点的前序信息聚合在一起，因此可以提高数据局部性；同理地，对于分发任务来说，分发任务要向所有后序结点提供信息，那么将分发任务分配给装载了出边模态的计算单元进行处理，能提高数据局部性，从而提高计算效率。
另外，在入边模态的子图和出边模态的子图之间，可通过小批量(英文：mini-batch)同步机制，为镜像节点提供信息更新。
在一些可能的实施例中，图计算装置可以获取任务与子图之间的调度方案，按照调度方案来并行执行多个算法的任务。具体来说，步骤603可以包括下述步骤A至步骤C：
步骤A、图计算装置获取多个算法的优先级。
在一些可能的实施例中，优先级的获取方式包括而不限于下述方式一至方式二中任一项或多项的结合：
方式一、图计算装置根据配置指令，获取多个算法的优先级，配置指令用于指示该多个算法的优先级。
在一些可能的实施例中，配置指令可以携带每个算法的重要性系数，可以根据配置指令，图计算装置可以获取每个算法的重要性系数，根据每个算法的重要性系数，获取每个算法的优先级。其中，算法的优先级可以与算法的重要性系数正相关。重要性系数表示算法的重要性，重要性系数越大，表示算法越重要，越需要被及时调度。重要性系数可以通过归一化，将取值范围控制在0至1的区间中。
方式二、图计算装置根据多个算法的迭代次数，获取多个算法的优先级。
图计算装置可以获取多个算法的迭代次数，对多个算法的迭代次数进行运算，得到多个算法的优先级。其中，算法的迭代次数为已采用算法对图进行计算的次数，即算法当前已执行的次数。算法的迭代次数可以小于或等于算法总共需要计算的次数。在一些可能的实施例中，如果期望新任务可以尽快得到结果，可以获取与迭代次数负相关的数值，作为算法的优先级，那么由于新任务的迭代次数较少，能够得到优先调度，从而尽快被计算单元执行。在另一些可能的实施例中，如果期望已经执行了较长时间的老任务可以尽快得到结果，可以获取与迭代次数正相关的数值，作为算法的优先级，那么由于老任务的迭代次数较多，能够得到优先调度，从而尽快执行。
作为示例，图计算装置可以按照下述公式(1)、公式(2)以及公式(3)中的任一项或多项的组合，对多个算法的重要性系数以及多个算法的迭代次数进行运算，得到多个算法的优先级。
a iA,jG＝δ iA,jG*ρ <A,G>*e -t<A,G>；(1)
a iA,jG＝δ iA,jG*ρ <A,G>*(1-e -t<A,G>)；(2)
a iA,jG＝1；(3)
其中，a iA,jG表示算法A对图G进行计算的优先级，ρ<A,G>为采用算法A对图G进行计算的重要性系数，ρ<A,G>进行了归一化，ρ<A,G>大于0且小于或等于1,t<A,G>为算法A当前对图G进行计算的迭代次数；δ<iA,jG>为狄拉克(Dirac Delta)函数，如果计算请求指示采用算法A对图G进行计算，则δ<iA,jG>取1，如果计算请求未指示采用算法A对图G进行计算，则δ<iA,jG>取0，*表示相乘。
在一些可能的实施例中，用户可以通过配置指令，控制新任务优先调度还是老任务优先调度。具体来说，如果用户期望新任务优先调度，可以触发用于指示图计算装置按照公式(1)运算的配置指令，图计算装置接收到配置指令后，会按照公式(1)来获取算法的优先级，使得算法的优先级与算法的迭代次数负相关。如果用户期望老任务能够得以优先调度，可以触发用于指示图计算装置按照公式(2)运算的配置指令，图计算装置接收到配置指令后，会按照公式(2)来获取算法的优先级，使得算法的优先级与算法的迭代次数正相关。如此，充分考虑了图被迭代算法执行时的场景，可以保证算法的优先级的定义方式可以由用户自定义设置，满足用户需求。也即是，有利于***根据不同的调度目标，实施调度，使得***具备良好的可配置能力。
步骤B、图计算装置根据多个算法中每个算法的优先级，获取调度方案。
调度方案用于指示至少一个目标任务以及至少一个目标子图之间的对应关系。例如，调度方案可以包括至少一个目标任务的标识以及至少一个目标子图的标识。其中，每个目标任务为多个算法中的任务，目标任务为多个算法的任务中本次调度的任务，目标子图为多个子图中的子图本次调度的子图。
算法的优先级用于指示算法在多个算法中的优先程度，每当要调度多个算法的任务时，可以先调度优先级高的多个算法的任务，后调度优先级低的多个算法的任务，从而控制不同算法的执行结束时间，让优先级高的算法能够尽快执行结束。
在一些可能的实施例中，调度方案的生成过程可以包括下述步骤(B.1)至步骤(B.3)：
步骤(B.1)图计算装置生成第一二分图。
第一二分图用于指示图和算法之间的调度关系，第一二分图可以视为粗粒度的二分图。第一二分图包括第一顶点、第二顶点以及第一边，第一顶点表示图，第二顶点表示算法，第一边连接第一顶点以及第二顶点，第一边表示当前接收到的计算请求中包括用于请求采用第二顶点对应的算法对第一顶点对应的图进行计算的计算请求。
示例性的，参见图11，第一二分图可以为图11中顶层的二分图，第一顶点为顶层的二分图中内侧的一排空心顶点，第二顶点为顶层的二分图中外侧的一排实心顶点，第一边为顶层的二分图中的实现。在图11的顶层的二分图中，存在与同一第一顶点连接的多个第二顶点，代表在图计算装置接收到的一批计算请求中，包括对同一图执行多个算法的请求，在此场景中，可以通过多算法并发执行，来提高计算速度。
步骤(B.2)图计算装置根据第一二分图，生成第二二分图。
第二二分图用于指示子图和任务之间的调度关系，第二二分图可以视为细粒度的二分图。包括第三顶点、第四顶点以及第二边，第三顶点表示任务，第四顶点表示子图，第二边连接第三顶点以及第四顶点。其中，结合子图的模态，第二二分图可以用于指示具有任一模态的子图和任务之间的调度关系，第四顶点表示某个模态的子图。 另外，第二边可以根据第一二分图的边来确定，例如，如果当前接收到的一批计算请求中包括采用算法i对图j进行计算的计算请求，则第一二分图中算法i对应的第一顶点与图j对应的第二顶点之间的第一边会得以连接，相应地，第二二分图中算法i的每个任务对应的第三顶点与图j的每个子图对应的第四顶点之间的第二边会连接。
示例性的，参见图11，第二二分图可以为图11中底层的二分图，第三顶点为底层的二分图中外侧的一排实心顶点，第四顶点为底层的二分图中内侧的一排空心顶点，第二边为底层的二分图中的实线。
在一些可能的实施例中，可以根据图与子图之间的对应关系，将第一顶点映射为第四顶点；根据算法与任务之间的对应关系，将第二顶点映射为第三顶点；将第一边映射为第二边，得到第二二分图。
示例性地，参见图11，图与子图之间的对应关系以及算法与任务之间的对应关系可以通过顶层的二分图与底层的二分图之间的层次关系来表示。具体地，图与子图之间的一条对应关系可以通过顶层的二分图与底层的二分图之间的一条内侧的虚线表示。例如，图A与子图B之间的对应关系可以通过顶层的二分图中内侧的顶点a与底层的二分图中内侧的顶点b之间的虚线表示。算法与任务之间的一条对应关系可以通过顶层的二分图与底层的二分图之间的一条外侧的虚线表示，例如，算法C与任务D之间的对应关系可以通过顶层的二分图中外侧的顶点c与底层的二分图中外侧的顶点d之间的虚线表示。
步骤(B.3)图计算装置从第二二分图中选择目标边，将目标边表示的任务与子图之间的对应关系，作为调度方案。
获取调度方案的过程可以建模为从底层的二分图中选择一组边的过程，在此不妨将选择出的边称为目标边。目标边连接的第三顶点为本次要调度的任务，目标边连接的第四顶点为本次要调度的子图。在一些可能的实施例中，考虑到一个任务最多分配到一个子图上，则一组目标边中同一个第三顶点最多连接一个第四顶点；另外，一个子图可以分配到一个或多个任务，则一组目标边中同一个第四顶点可以连接一个或多个第三顶点。
作为示例，选择目标边的过程可以包括下述步骤(B.3.1)至步骤(B.3.2)。
步骤(B.3.1)对于第一二分图中的每条边，图计算装置根据边对应的子图以及任务，获取边的权重。
步骤(B.3.1)可以包括下述方式一至方式二中任一项或多项的结合。
方式一、获取边对应的重要性系数，根据重要性系数，获取边的权重。
例如，如果边连接的第一顶点表示图G，边连接的第二顶点表示算法A，可以获取采用算法A对图G进行计算的重要性系数，对重要性系数进行运算，得到权重，其中权重与重要性系数正相关。
方式二、获取边对应的迭代次数，根据迭代次数，获取边的权重。
例如，如果边连接的第一顶点表示图G，边连接的第二顶点表示算法A，可以获取已采用算法A对图G进行计算的次数，对迭代次数进行运算，得到权重。
需要说明的一点是，方式一和方式二可以择一执行也可以结合执行。例如，如果方式一和方式二结合，可以获取边对应的重要性系数以及迭代次数，对重要性系数以 及迭代次数进行运算，得到边的权重。例如，可以采用上述公式(1)以及公式(2)中的任一项对重要性系数以及迭代次数进行运算。
步骤(B.3.2)图计算装置根据第一二分图中每条边的权重，从第一二分图中选择目标边，目标边的权重符合预设条件。
该预设条件可以包括权重之和在第二二分图中的多个边的权重之和中最大、权重之和在第二二分图中的多个边的权重之和中排在前预设位数、权重之和超过阈值等。
在一些可能的实施例中，图计算装置可以利用约束条件，采用线性规划的方法来选择目标边。具体来说，图计算装置可以根据第一二分图，获取目标函数；采用线性规划算法，根据约束条件对目标函数进行运算，得到目标函数的解，作为目标边。
其中，目标函数用于从第一二分图中选择目标边，例如，目标函数可以指示求解权重之和最大的一组边。示例性地，目标函数可以如下式(4)所示。
max(TrB TX)；(4)
其中，max表示求最大值，Tr表示对矩阵求迹(trace)，B是重要性矩阵A的转写，B＝c+log 2|A|。c是常数，通过B和c可以用于矩阵的预处理(precodnition)技术，能获取数值计算上的好处。A是重要性系数矩阵，A中的每一行表示一个算法分解后得到的任务，A中的每一列表示一个模态的一个子图，A中的每一个非零元素表示元素对应的任务以及元素对应的子图时能够分配到一起，且该非零元素的数值为对子图执行任务的重要性系数。X是全排列矩阵(premutation matric)，X的作用是对B T的列进行交换，使得交换后的矩阵的对角线上的元素之和最大，即找到一组最优解。对角线上元素表示的任务与子图之间的对应关系，即为选择出来的调度方案。
约束条件用于保证目标边中的不同边之间不交连(disjoint)，例如，约束条件可以用于约束上述公式(4)求解的X中每行有且只有一个1，每列有且只有一个1。作为示例，约束条件可以包括下式(5)、下式(6)以及下式(7)，其中，T表示转置运算。
Figure PCTCN2019125798-appb-000001
Figure PCTCN2019125798-appb-000002
Figure PCTCN2019125798-appb-000003
线性规划算法可以是二分图的子图匹配算法，例如可以是拍卖算法(英文：auction algorithm)、匈牙利路径增强算法(英文：Hungarian augmented path algorithm)等。
通过运行上述线性规划算法，可以得到上式(4)中的X，X即为第二二分图中的一组不交连的边，X正好给出一组调度方案：把第二二分图中的某一条边中一端的顶点(表示任务)与另一端的顶点(表示某个模态的某个子图)调度在一起执行。
通过上述实现方式，对于在同一张图上执行的多个算法，其关联关系会在粗粒度的算法与图之间的第一二分图上体现，进而体现到细粒度的第二二分图中，因此在按照上述基于线性规划的方法实施调度的时候，一个模态的一个子图能够被多个任务并 行处理，而不会被反复加载和退出(evict)。
步骤C、图计算装置通过加载了至少一个目标子图的多个计算单元，并行执行至少一个目标任务。
图计算装置可以根据调度方案，确定本次调度的目标任务以及目标子图，确定加载了目标子图的计算单元，通过这些加载了目标子图的计算单元，来并行执行目标任务。在一些可能的实施例中，当计算单元并行执行目标任务结束后，图计算装置可以重复执行上述步骤A至步骤C，即重新获取调度方案，以得到剩余的任务与子图之间的对应关系，从而分配剩余的任务，直至多个算法的任务均已执行结束。
参见图12，其示出了图计算方法的示例性的流程，图12中左侧的分支可以在离线时预先执行，包括下述步骤(1)至步骤(4)：
(1)用户编辑图的模式信息：
(2)根据模式信息导入原始数据，根据原始数据构建图；
(3)将输入的图划分为多个子图；
(4)对每个模态的每个子图进行持久化存储。
图12右侧的分支可以在在线时实时执行，包括下述步骤(5)至步骤(10)：
(5)接收批量的计算请求：
(6)生成第一二分图；
(7)将算法分解为任务，加载每个模态的每个子图；
(8)基于任务以及子图，生成第二二分图；
(9)求解二分图，得到调度方案；
(10)根据调度方案，加载子图，执行任务。
考虑到对图执行的算法大多为迭代算法，参见图12，采用迭代算法对图进行计算的过程还可以包括下述步骤一至步骤三：
步骤一、计算单元根据多个子图，执行迭代算法的第一次执行过程的任务，得到第一次执行过程的计算结果。
步骤二、计算单元判断算法是否收敛。
作为示例，计算单元可以根据第一次执行过程的计算结果以及迭代次数中的至少一项，判断迭代算法尚未收敛。例如，计算单元可以判断迭代次数是否达到预设次数，如果迭代次数未达到预设次数，则计算单元确定迭代算法尚未收敛，如果迭代次数已达到预设次数，则计算单元确定迭代算法收敛。又如，计算单元可以判断第一次执行过程的计算结果是否符合预设条件，如果第一次执行过程的计算结果未符合预设条件，则计算单元确定迭代算法尚未收敛。
步骤三、如果迭代算法未收敛，计算单元根据多个子图以及第一次执行过程的计算结果，执行迭代算法的第二次执行过程的任务，得到第二次执行过程的计算结果。
之后，计算单元可以重新判断迭代算法是否收敛，如果迭代算法尚未收敛，则根据多个子图以及第二次执行过程的计算结果，执行迭代算法的第三次执行过程的任务，依次类推，直至迭代算法收敛，可以输出本次执行过程的计算结果，例如对本次执行过程的计算结果进行可视化，呈现本次执行过程的计算结果。
在一些可能的实施例中，参见图12，对于多个算法中尚未收敛的算法来说，可以 获取尚未收敛的算法的任务，作为剩余任务，从而动态生成新的任务。如果接收到下一次计算请求，可以获取下一次计算请求中请求的算法的任务，将这些新的任务与剩余任务一起并发执行，直至没有新的计算请求需要处理。
在一些可能的实施例中，在多个计算单元并行执行多个算法的过程中，图计算装置可以对多个计算单元进行扩容，以便通过更多的计算单元来执行算法，具体来说，扩容的过程详见下述步骤一至步骤八。
步骤一、图计算装置向终端发送扩容请求。
扩容请求用于请求对多个计算单元进行扩容。在一些可能的实施例中，扩容请求可以包括要新增的计算单元的数量以及扩容费用中的至少一项，扩容费用为用户要为新增的计算单元支付的费用。例如，如果单位时长内租用一个计算单元需要支付n，而当前要新增m个计算单元，则扩容请求可以包括要新增的计算单元的数量m以及扩容费用(m*n)，其中m为正整数，n为正数。
在一些可能的实施例中，图计算装置可以在多个计算单元的计算能力不满足需求时，生成扩容请求，向终端发送扩容请求。例如，终端向图计算装置发送的计算请求可以包括期望时长，该期望时长为用户期望多个计算单元完成对图计算的时长，图计算装置可以判断多个计算单元是否能够在期望时长内完成对图进行计算，如果多个计算单元不能够在期望时长内完成对图进行计算，图计算装置可以生成扩容请求。又如，图计算装置可以监控每个计算单元的负载情况，实时判断每个计算装置的负载情况是否超过阈值，当一个或多个计算单元的负载情况超过阈值时，图计算装置可以生成扩容请求。又如，多个计算单元可以监控自身的负载情况是否超过阈值，如果自身的负载情况超过阈值，则多个计算单元向图计算装置发送通知消息，图计算装置接收到一个或多个计算单元的通知消息后，图计算装置可以生成扩容请求。又如，图计算装置可以根据图的数据量以及每个计算单元的规格，判断多个计算单元能够计算的总数据量是否大于图的数据量，如果多个计算单元能够计算的总数据量小于或等于图的数据量，图计算装置可以生成扩容请求。
步骤二、终端接收图计算装置的扩容请求，显示提示信息。
提示信息用于提示用户是否对多个计算单元进行扩容。在一些可能的实施例中，该提示信息还可以包括要新增的计算单元的数量以及扩容费用中的至少一项。
步骤三、终端接收确认指令，生成扩容指令，向图计算装置发送扩容指令，扩容指令用于指示对多个计算单元进行扩容。
确认指令用于指示确认对多个计算单元进行扩容，确认指令可以通过用户在终端上执行的确认操作触发。作为示例，终端显示的提示信息可以包括确认选项，如果用户对确认选项触发点击操作，则终端接收到确认指令。
步骤四、图计算装置接收终端的扩容指令。
在一些可能的实施例中，终端接收确认指令后，可以向金融服务器发送支付请求，支付请求包括扩容费用、用户账户的标识以及图计算装置对应的商家账户的标识，金融服务器接收到支付请求后，会从支付请求中获取用户账户的标识以及商家账户的标识，从用户账户中扣除扩容费用，向商家账户中增加扩容费用；扣款成功后，金融服务器可以向终端发送支付成功消息，终端接收到支付成功消息后，向图计算装置发送 扩容指令，图计算装置接收扩容指令，执行下述步骤五；或者，金融服务器可以向图计算装置发送支付成功消息，图计算装置接收到支付成功消息且接收到扩容指令后，执行下述步骤五。
需要说明的一点是，上述仅是以图计算装置和金融服务器为两个分离的装置为例进行描述，在另一些可能的实施例中，图计算装置和金融服务器可以集成在一起，该集成的装置同时具有图计算的功能以及在线交易的功能。在这种实现方式中，该集成的装置可以预存有商家账户的标识，终端接收确认指令后，可以向该集成的装置发送支付请求，该集成的装置接收到支付请求后，会从用户账户中扣除扩容费用，向商家账户中增加扩容费用；扣款成功后，该集成的装置可以执行下述步骤五。
步骤五、图计算装置创建至少一个计算单元。
步骤六、图计算装置对图的至少一个子图进行复制，得到至少一个子图的实例。
其中，子图的实例是子图的拷贝，子图的实例可以和子图相同。
步骤七、图计算装置将至少一个子图的实例加载到创建的至少一个计算单元。
步骤七与上述步骤602同理，在此不做赘述。
步骤八、图计算装置通过该创建的至少一个计算单元，并行执行多个算法。
步骤八与上述步骤603同理，在此不做赘述。
在一些可能的实施例中，图计算装置可以统计图中每个子图被多个算法请求的次数，判断每个子图的请求次数是否达到阈值，对图中请求次数达到阈值的子图进行复制，得到请求次数达到阈值的子图的实例，将这些子图的实例加载到扩容出的目标计算单元，由目标计算单元在这些子图的实例上并行执行多个算法。其中，请求次数为多个算法请求子图的次数之和。
通过这种实现方式，考虑到单个图能够支持的访问数量具有物理上限的，如果图的访问数量超过物理上限，很可能在图上无法支持并行执行多个算法，也就对图算法的执行过程造成瓶颈。例如，如果某个子图是热点子图，比如说，该子图是包含了名人信息的子图，同一时刻很可能有大量用户访问该子图以查询名人信息，则该子图可能无法支持被多个任务同时调度，也就无法并行执行多个算法，导致对图计算的整体速度造成限制。而上述实现方式中，图计算装置能够感知每个子图被请求的数量，如果某个子图的请求次数超过阈值，表明该子图的需求量很大，很可能是热点子图，那么通过触发该子图被复制为多份，分别部署在不同的计算单元上，由多个计算单元对该子图进行处理，能够提高这个子图的处理效率。也即是，通过让并发的计算请求能够分流至不同的实例上来进行计算，从而线性扩展了并发性。
在一些可能的实施例中，在并行执行多个算法的过程中，可以对多个子图进行写保护。作为示例，写保护的方式包括而不限于下述方式一至方式三中的任一项或多项的结合：
方式一(加锁)、当任一算法的任一任务写入任一子图时，计算单元可以对子图加锁，在执行该任务的期间，除了该任务以外的其他任务可以无法写入该子图。当算法的任务对子图写入结束后，计算单元可以释放锁。通过执行方式一，一方面，可以保证对子图执行的写入步骤得以串行执行，另一方面，这种方式在实现和管理起来较为简单。
方式二(多版本并发控制(英文全称：multi-version concurrency control，英文简称：MVCC))。计算单元可以缓存多个子图中每个子图的全态；例如，可以存储子图的当前态、历史态以及过渡态。通过执行方式二，有利于同时进行写的操作。
方式三、对于多个子图中的任一子图，可以从子图的多个实例中选择目标实例，该目标实例可以视为多个实例中的首选实例。当接收到对子图的任一实例的写指令时，可以根据写指令，先向目标实例写入数据，当写入结束时，对目标实例与多个实例中目标实例以外的其他实例进行数据同步，从而实现子图的所有实例的数据一致性。可选地，当接收到对任一实例的写指令时，可以终止分发任务，待所有实例完成写操作后方可恢复，从而实现更严格的一致。
通过上述实现方式，可以降低保证多个任务共享同一子图时，发生数据冲突的概率，保证分布式计算单元并行执行多任务时***的稳健性。
在一些可能的实施例中，图计算装置可以为每个算法的每个任务创建缓存空间，当得到该任务的任一中间结果时，图计算装置可以将中间结果缓存在该缓存空间中，当该任务执行结束后，可以释放该缓存空间。
本实施例提供的方法，提供了一种能够支持多算法并发执行图计算的方法，通过将图的多个子图加载到多个计算单元，通过多个计算单元，并行执行多个算法，可以在多个算法之间共享同一份图，从而在同一份图上并行执行多个算法，节省了执行某个算法时需要等待其他算法执行结束所造成的时延，从而提高了多算法进行图计算的整体效率，缩短了多算法进行图计算的整体时间。
本申请还提供了一种图计算装置1300。如图13所示，图计算装置1300包括接收单元1301，加载单元1302以及多个计算单元1303。
接收单元1301用于执行步骤601。加载单元1302用于执行步骤602。多个计算单元1303用于执行步骤603。
在一种可能的实现中，多个计算单元1303，包括：获取模块，用于执行步骤603中的步骤一，例如可以用于执行步骤603中的步骤一中的方式一至方式九中的任一项或多项的组合，以将算法划分为多个任务；执行模块，用于执行步骤603中的步骤二。
在一种可能的实现中，加载单元1302，用于将至少一种模态的多个子图加载到多个计算单元；执行模块，用于根据目标模态的子图执行任务；
在一种可能的实现中，执行模块，用于执行步骤603中的步骤A至步骤C。
在一种可能的实现中，多个计算单元1303还用于执行算法中的加载步骤以外的部分。
在一种可能的实现中，多个计算单元1303还用于执行算法中的释放步骤以外的部分。
在一种可能的实现中，该装置还包括：划分单元，用于对图进行划分；存储单元，用于将多个子图存入图存储装置；加载单元1302，用于将图的多个子图从图存储装置加载到多个计算单元。
在一种可能的实现中，划分单元，用于根据多个计算单元1303的数量，对图进行划分。
在一种可能的实现中，装置还包括发送单元，用于发送扩容请求；接收单元，还 用于接收扩容指令；创建单元，用于创建至少一个计算单元；复制单元，用于对图的至少一个子图进行复制；加载单元1302，还用于将至少一个子图的实例加载到创建的至少一个计算单元；创建的至少一个计算单元，用于并行执行多个算法。
图计算装置1300可以作为云计算服务向用户提供，例如作为图引擎服务向用户提供。例如图1所示，图计算装置1300(或其部分)部署在云环境上，用户通过终端触发计算请求后，启动图计算装置1300对图执行多个算法，计算结果可以被提供给用户。
需要说明的一点是，图13实施例提供的图计算装置在进行图计算时，仅以上述各单元以及模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的单元以及模块完成，即将图计算装置的内部结构划分成不同的单元以及模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的图计算装置与图计算方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
本申请还提供了一种图计算装置1400。如图14所示，图计算装置1400包括多个计算单元1410，每个计算单元1410包括处理器1411、存储器1412、收发器1413以及总线1414，处理器1411、存储器1412、收发器1413之间通过总线1414通信。
其中，处理器可以为中央处理器(英文：central processing unit，缩写：CPU)。存储器可以包括易失性存储器(英文：volatile memory)，例如随机存取存储器(英文：random access memory，缩写：RAM)。存储器还可以包括非易失性存储器(英文：non-volatile memory)，例如只读存储器(英文：read-only memory，缩写：ROM)，快闪存储器，HDD或SSD。存储器中存储有可执行代码，处理器执行该可执行代码以执行前述图计算方法。存储器中还可以包括操作***等其他运行进程所需的软件模块。操作***可以为LINUX TM,UNIX TM,WINDOWS TM等。
图计算装置1400的每个计算单元1410的存储器1412中存储了图计算装置1300的各个单元对应的代码，处理器1411执行这些代码实现了图计算装置1300的各个单元的功能，即执行了图6实施例提供的图计算方法。
图计算装置1400可以部署在云环境中，图计算装置1400中的多个计算单元1410可以组成分布式***，不同计算单元1410可以通过有线或无线网络进行通信。
本申请还提供了一种图计算装置1500。如图15所示，图计算装置1500包括多个计算单元1510、收发器1511以及总线1512；
每个计算单元1510包括处理器1521以及存储器1522；其中，处理器1521可以与处理器1411同理，存储器1522可以与存储器1412同理，在此不做赘述。
图计算装置1500中的多个计算单元1510可以部署在单机中，不同计算单元1510可以通过总线1512进行通信。
本申请还提出了一种图计算***，如图16所示，该图计算***在包括图14实施例所示的图计算装置1400的基础上，还包括图存储装置1600，图存储装置1600用于存储图的多个子图。在一些可能的实施例中，图存储装置1600可以作为云存储服务向用户提供，用户可以在该云存储服务中申请一定容量的存储空间，在该存储空间中存储图的多个子图。图计算装置1400运行时，通过通信网络从图存储装置1600中加载所需的多个子图。
作为一种可能的产品形态，本申请实施例的图计算装置可以由通用处理器来实现。 通用处理器包括处理电路和与处理电路内部连接通信的输入接口，输入接口用于接收至少一个计算请求；处理电路用于执行上述图计算方法。可选地，该通用处理器还可以包括存储介质，存储介质用于存储处理电路执行的指令。
作为一种可能的产品形态，本申请实施例的图计算装置，还可以使用下述来实现：一个或多个现场可编程门阵列(英文全称：field－programmable gate array，英文简称：FPGA)、可编程逻辑器件(英文全称：programmable logic device，英文简称：PLD)、控制器、状态机、门逻辑、分立硬件部件、任何其它适合的电路、或者能够执行本申请通篇所描述的各种功能的电路的任意组合。
在一些可能的实施例中，本申请还提供了一种计算机程序产品，该计算机程序产品被图计算装置执行时，该图计算装置执行上述图计算方法。该计算机程序产品可以为一个软件安装包，在需要使用前述图计算方法的情况下，可以下载该计算机程序产品并在图计算装置上执行该计算机程序产品。
上述各个附图对应的流程的描述各有侧重，某个流程中没有详述的部分，可以参见其他流程的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例中描述的各方法步骤和单元，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各实施例的步骤及组成。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域普通技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的***、装置和单元的具体工作过程，可以参见前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本申请实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的 形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如SSD)等。
应理解，上述各种产品形态的图计算装置，分别具有上述图6方法实施例中图计算方法的任意功能，此处不再赘述。
以上该，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (21)

  1. 一种图计算方法，其特征在于，所述方法包括：
    接收至少一个计算请求，所述至少一个计算请求用于请求采用多个算法对图进行计算；
    将所述图的多个子图加载到多个计算单元；
    通过所述多个计算单元，并行执行所述多个算法。
  2. 根据权利要求1所述的方法，其特征在于，所述通过所述多个计算单元，并行执行所述多个算法，包括：
    获取每个算法中的至少一个任务；
    通过所述多个计算单元，并行执行所述多个算法中的任务。
  3. 根据权利要求2所述的方法，其特征在于，所述获取每个算法中的至少一个任务，包括下述至少一项：
    根据所述算法中每个步骤对应的函数名称，将所述算法中同一函数名称对应的至少一个步骤划分为一个任务；
    根据所述算法中每个步骤的执行主体，将所述算法中执行主体相同的步骤划分为一个任务；
    根据所述算法中每个步骤对图中顶点或边的访问顺序，将所述算法中访问顺序相同的步骤划分为一个任务；
    根据所述算法中每个步骤在图中访问的顶点或边，将所述算法中访问的顶点或边相同的步骤划分为一个任务；
    根据所述算法中每个步骤执行的动作，将所述算法中动作相同的步骤划分为一个任务；
    根据所述算法中每个步骤所属的迭代过程，将所述算法中属于同一次迭代过程的步骤划分为一个任务；
    根据所述算法中每个步骤所属的判断分支，将所述算法中属于同一判断分支的步骤划分为一个任务。
  4. 根据权利要求2所述的方法，其特征在于，
    所述将所述图的多个子图加载到多个计算单元，包括：
    将至少一种模态的所述多个子图加载到所述多个计算单元；
    所述通过所述多个计算单元，并行执行所述多个算法中的任务，包括：
    对于所述多个算法的任一任务，通过所述多个计算单元，根据目标模态的子图执行所述任务，所述目标模态为所述至少一种模态中与所述任务匹配的模态。
  5. 根据权利要求4所述的方法，其特征在于，所述任务包括从所述图中第一顶点搜索至所述图中第二顶点的步骤，所述目标模态的子图中所述第二顶点排在所述第一顶点之前。
  6. 根据权利要求2所述的方法，其特征在于，所述通过所述多个计算单元，并行执行所述多个算法中的任务，包括：
    根据所述多个算法的迭代次数，获取所述多个算法的优先级；
    根据每个算法的优先级，获取调度方案，所述调度方案用于指示至少一个目标任 务以及至少一个目标子图之间的对应关系，所述目标任务为所述多个算法的任务中本次调度的任务，所述目标子图为所述多个子图中的子图本次调度的子图；
    通过加载了所述至少一个目标子图的多个计算单元，并行执行所述至少一个目标任务。
  7. 根据权利要求1至6中任意一项所述的方法，其特征在于，所述通过所述多个计算单元，并行执行所述多个算法，包括下述至少一项：
    对于所述多个算法中的任一算法，执行所述算法中的加载步骤之外的部分，所述加载步骤为将所述图加载到计算单元的内存的步骤；
    对于所述多个算法中的任一算法，执行所述算法中的释放步骤之外的部分，所述释放步骤为将所述图从计算单元的内存中释放的步骤。
  8. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    发送扩容请求，所述扩容请求用于请求对所述多个计算单元进行扩容；
    接收扩容指令，所述扩容指令用于指示对所述多个计算单元进行扩容；
    创建至少一个计算单元；
    对所述图的至少一个子图进行复制，得到至少一个子图的实例；
    将所述至少一个子图的实例加载到所述创建的至少一个计算单元；
    通过所述创建的至少一个计算单元，并行执行所述多个算法。
  9. 根据权利要求8所述的方法，其特征在于，所述对所述图的至少一个子图进行复制，包括：
    统计所述图中每个子图被所述多个算法请求的次数；
    对所述图中请求次数达到阈值的子图进行复制。
  10. 一种图计算装置，其特征在于，包括：
    接收单元，用于接收至少一个计算请求，所述至少一个计算请求用于请求采用多个算法对图进行计算；
    加载单元，用于将所述图的多个子图加载到多个计算单元；
    所述多个计算单元，用于通过所述多个计算单元，并行执行所述多个算法。
  11. 根据权利要求10所述的装置，其特征在于，所述多个计算单元，包括：
    获取模块，用于获取每个算法中的至少一个任务；
    执行模块，用于通过所述多个计算单元，并行执行所述多个算法中的任务。
  12. 根据权利要求11所述的装置，其特征在于，所述获取模块，用于执行下述至少一项：
    根据所述算法中每个步骤对应的函数名称，将所述算法中同一函数名称对应的至少一个步骤划分为一个任务；
    根据所述算法中每个步骤的执行主体，将所述算法中执行主体相同的步骤划分为一个任务；
    根据所述算法中每个步骤对图中顶点或边的访问顺序，将所述算法中访问顺序相同的步骤划分为一个任务；
    根据所述算法中每个步骤在图中访问的顶点或边，将所述算法中访问的顶点或边相同的步骤划分为一个任务；
    根据所述算法中每个步骤执行的动作，将所述算法中动作相同的步骤划分为一个任务；
    根据所述算法中每个步骤所属的迭代过程，将所述算法中属于同一次迭代过程的步骤划分为一个任务；
    根据所述算法中每个步骤所属的判断分支，将所述算法中属于同一判断分支的步骤划分为一个任务。
  13. 根据权利要求11所述的装置，其特征在于，
    所述加载单元，用于将至少一种模态的所述多个子图加载到所述多个计算单元；
    所述执行模块，用于对于所述多个算法的任一任务，通过所述多个计算单元，根据目标模态的子图执行所述任务，所述目标模态为所述至少一种模态中与所述任务匹配的模态。
  14. 根据权利要求13所述的装置，其特征在于，所述任务包括从所述图中第一顶点搜索至所述图中第二顶点的步骤，所述目标模态的子图中所述第二顶点排在所述第一顶点之前。
  15. 根据权利要求11所述的装置，其特征在于，所述执行模块，用于根据所述多个算法的迭代次数，获取所述多个算法的优先级；根据每个算法的优先级，获取调度方案，所述调度方案用于指示至少一个目标任务以及至少一个目标子图之间的对应关系，所述目标任务为所述多个算法的任务中本次调度的任务，所述目标子图为所述多个子图中的子图本次调度的子图；通过加载了所述至少一个目标子图的多个计算单元，并行执行所述至少一个目标任务。
  16. 根据权利要求10至15中任意一项所述的装置，其特征在于，所述多个计算单元，用于执行下述至少一项：对于所述多个算法中的任一算法，执行所述算法中的加载步骤之外的部分，所述加载步骤为将所述图加载到计算单元的内存的步骤；对于所述多个算法中的任一算法，执行所述算法中的释放步骤之外的部分，所述释放步骤为将所述图从计算单元的内存中释放的步骤。
  17. 根据权利要求10所述的装置，其特征在于，所述装置还包括：
    发送单元，用于发送扩容请求，所述扩容请求用于请求对所述多个计算单元进行扩容；
    所述接收单元，还用于接收扩容指令，所述扩容指令用于指示对所述多个计算单元进行扩容；
    创建单元，用于创建至少一个计算单元；
    复制单元，用于对所述图的至少一个子图进行复制，得到至少一个子图的实例；
    所述加载单元，还用于将所述至少一个子图的实例加载到所述创建的至少一个计算单元；
    所述创建的至少一个计算单元，用于并行执行所述多个算法。
  18. 根据权利要求17所述的装置，其特征在于，所述复制单元，用于统计所述图中每个子图被所述多个算法请求的次数；对所述图中请求次数达到阈值的子图进行复制。
  19. 一种图计算装置，其特征在于，包括多个计算单元，每个计算单元包括处理 器和存储器，所述存储器中存储有至少一条指令，所述指令由所处理器加载并执行以实现如权利要求1至9中任一所述的方法。
  20. 一种图计算***，其特征在于，包括图计算装置和图存储装置；
    所述图计算装置包括多个计算单元；
    所述图存储装置用于存储图的多个子图；
    所述图计算装置用于将所述多个子图从所述图存储装置加载至所述多个计算单元，以执行权利要求1至9中任一所述的方法。
  21. 一种非瞬态的可读存储介质，其特征在于，所述非瞬态的可读存储介质被图计算装置执行时，所述图计算装置执行权利要求1至9中任一所述的方法。
PCT/CN2019/125798 2019-04-24 2019-12-17 图计算方法及装置 WO2020215752A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19925708.0A EP3955132A4 (en) 2019-04-24 2019-12-17 GRAPH CALCULATION METHOD AND DEVICE
US17/508,294 US20220043675A1 (en) 2019-04-24 2021-10-22 Graph computing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910335121.4 2019-04-24
CN201910335121.4A CN111859027A (zh) 2019-04-24 2019-04-24 图计算方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/508,294 Continuation US20220043675A1 (en) 2019-04-24 2021-10-22 Graph computing method and apparatus

Publications (1)

Publication Number Publication Date
WO2020215752A1 true WO2020215752A1 (zh) 2020-10-29

Family

ID=72940709

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/125798 WO2020215752A1 (zh) 2019-04-24 2019-12-17 图计算方法及装置

Country Status (4)

Country Link
US (1) US20220043675A1 (zh)
EP (1) EP3955132A4 (zh)
CN (1) CN111859027A (zh)
WO (1) WO2020215752A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111309479B (zh) * 2020-02-14 2023-06-06 北京百度网讯科技有限公司 一种任务并行处理的实现方法、装置、设备和介质
US12032632B2 (en) * 2020-11-20 2024-07-09 International Business Machines Corporation DFS-based cycle detection on Pregel model
CN112506657B (zh) * 2020-12-09 2022-04-08 上海交通大学 基于强化学习的面向微服务的资源管理***
CN113610234A (zh) * 2021-08-11 2021-11-05 中国银行股份有限公司 基于深度优先搜索的优化算法的全排列方法及装置
CN113900786A (zh) * 2021-10-11 2022-01-07 国网电子商务有限公司 一种基于图数据的分布式计算方法及装置
CN114282073B (zh) * 2022-03-02 2022-07-15 支付宝(杭州)信息技术有限公司 数据存储方法及装置、数据读取方法及装置
US20240202814A1 (en) * 2022-12-14 2024-06-20 International Business Machines Corporation Graph feature based system for flow management

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102841816A (zh) * 2012-07-16 2012-12-26 北京市遥感信息研究所 遥感图像数据并行处理的方法
CN103532710A (zh) * 2013-09-26 2014-01-22 中国科学院数据与通信保护研究教育中心 基于gpu的sm2算法实现方法及装置
CN103793442A (zh) * 2012-11-05 2014-05-14 北京超图软件股份有限公司 空间数据的处理方法及***
CN106445688A (zh) * 2016-09-30 2017-02-22 电子科技大学 一种基于mic计算平台的nlm并行图像增强方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104780213B (zh) * 2015-04-17 2018-02-23 华中科技大学 一种主从分布式图处理***负载动态优化方法
CN105653204B (zh) * 2015-12-24 2018-12-07 华中科技大学 一种基于磁盘的分布式图计算方法
CN108364327B (zh) * 2017-01-26 2021-09-17 华为技术有限公司 一种图数据处理的方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102841816A (zh) * 2012-07-16 2012-12-26 北京市遥感信息研究所 遥感图像数据并行处理的方法
CN103793442A (zh) * 2012-11-05 2014-05-14 北京超图软件股份有限公司 空间数据的处理方法及***
CN103532710A (zh) * 2013-09-26 2014-01-22 中国科学院数据与通信保护研究教育中心 基于gpu的sm2算法实现方法及装置
CN106445688A (zh) * 2016-09-30 2017-02-22 电子科技大学 一种基于mic计算平台的nlm并行图像增强方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3955132A4 *

Also Published As

Publication number Publication date
EP3955132A4 (en) 2022-06-01
EP3955132A1 (en) 2022-02-16
US20220043675A1 (en) 2022-02-10
CN111859027A (zh) 2020-10-30

Similar Documents

Publication Publication Date Title
WO2020215752A1 (zh) 图计算方法及装置
CN106888254B (zh) 一种基于Kubernetes的容器云架构及其各模块之间的交互方法
US11580107B2 (en) Bucket data distribution for exporting data to worker nodes
US11341131B2 (en) Query scheduling based on a query-resource allocation and resource availability
US11599541B2 (en) Determining records generated by a processing task of a query
US11373173B2 (en) Distributed ledger system, distributed ledger subsystem, and distributed ledger node
CN109976667B (zh) 一种镜像管理方法、装置及***
CN108431796B (zh) 分布式资源管理***和方法
US20190258636A1 (en) Record expansion and reduction based on a processing task in a data intake and query system
US20190258632A1 (en) Determining a Record Generation Estimate of a Processing Task
US8788760B2 (en) Adaptive caching of data
CN106919445B (zh) 一种在集群中并行调度容器的方法和装置
CN105094997B (zh) 一种云计算主机节点间物理内存共享方法和***
Du et al. Scientific workflows in IoT environments: a data placement strategy based on heterogeneous edge-cloud computing
CN105531688B (zh) 提供作为其它服务的资源的服务
CN111901294A (zh) 一种构建在线机器学习项目的方法及机器学习***
US20110145153A1 (en) Negotiating agreements within a cloud computing environment
CN104160381A (zh) 多租户环境中租户特定数据集的管理
US8874587B2 (en) Tenant placement in multitenant cloud databases with one-to-many data sharing
CN109117252B (zh) 基于容器的任务处理的方法、***及容器集群管理***
CN106856438B (zh) 一种网络业务实例化的方法、装置及nfv***
CN104050042A (zh) Etl作业的资源分配方法及装置
CN104754008A (zh) 网络存储节点、网络存储***以及用于网络存储节点的装置和方法
US8832156B2 (en) Distributed computing management
CN108696559B (zh) 流处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19925708

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019925708

Country of ref document: EP

Effective date: 20211109