CN113918305A - Node scheduling method and device, electronic equipment and readable storage medium - Google Patents
Node scheduling method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN113918305A CN113918305A CN202111270895.7A CN202111270895A CN113918305A CN 113918305 A CN113918305 A CN 113918305A CN 202111270895 A CN202111270895 A CN 202111270895A CN 113918305 A CN113918305 A CN 113918305A
- Authority
- CN
- China
- Prior art keywords
- node
- plan information
- nodes
- screening
- time interval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012216 screening Methods 0.000 claims abstract description 51
- 238000004590 computer program Methods 0.000 claims description 14
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000007726 management method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9532—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a load dispatching technology, and discloses a node scheduling method, which comprises the following steps: screening the node plan information set according to the plan information of each node to obtain a target node plan information set; screening all nodes according to the node IP and the node IP in the target node plan information set, and marking each screened node by using node grouping information to obtain a first node set; node screening is carried out on the first node set according to the node state of each node in the first node set, and a second node set is obtained; and responding to the query task request, and selecting nodes in the second node set according to the user service label and the grouping label to execute the query task. The invention also relates to a blockchain technique, the node plan information set may be stored in blockchain link points. The invention also provides a node scheduling device, equipment and a medium. The invention can improve the flexibility of node scheduling.
Description
Technical Field
The present invention relates to load scheduling technologies, and in particular, to a node scheduling method and apparatus, an electronic device, and a readable storage medium.
Background
presto is a type of query engine that can provide responses on the order of seconds or even milliseconds in conjunction with hadoop for large data volume queries.
However, the native presto cannot perform physical resource isolation on the queries, so that queries of different services can affect each other if the queries are distributed to the same node, and the flexibility of node scheduling and allocation is low.
Disclosure of Invention
The invention provides a node scheduling method, a node scheduling device, electronic equipment and a computer-readable storage medium, and mainly aims to improve the flexibility of node scheduling.
In order to achieve the above object, a node scheduling method provided by the present invention includes:
acquiring a node plan information set, wherein each node plan information in the node plan information set comprises: node IP, effective time interval, node grouping information and node preset state;
acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
dynamically acquiring all running nodes and corresponding running node IPs in a presto cluster, carrying out IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking grouping labels on each screened node by using the node grouping information to obtain a first node set;
identifying the node state of each node in the first node set, and screening the nodes of the first node set according to the result of the node state identification to obtain a second node set;
and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
Optionally, the screening the node plan information set according to the valid time interval, the preset node state, and the task valid time interval to obtain a target node plan information set includes:
extracting node plan information of which the preset state of the node is an alive state in the node plan information set to obtain an initial node plan information set;
and screening the node plan information of the effective time interval in the task effective time interval in the initial node plan information set to obtain a target node plan information set.
Optionally, the performing IP matching screening on all the nodes according to each node IP in the target node plan information set and the operating node IP, and performing packet tag marking on each screened node by using the node packet information to obtain a first node set includes:
screening nodes corresponding to the operating node IP which is the same as the node IP;
selecting node grouping information corresponding to the node IP which is the same as the running node IP in the target node plan information set to obtain target node grouping information;
grouping label marking is carried out on the corresponding nodes by utilizing the target node grouping information;
and summarizing all the nodes marked by the labels to obtain the first node set.
Optionally, the performing node state identification on each node in the first node set, and performing node screening on the first node set according to a result of the node state identification to obtain a second node set includes:
acquiring the node state of each node in the first node set;
and screening the nodes with the node states of active in the first node set to obtain the second node set.
Optionally, the selecting a node in the second node set according to the user service tag and the group tag to execute a query task includes:
screening the corresponding grouping labels in the second node set as the nodes of the user service labels to obtain a target node set;
and executing the query task by utilizing the nodes in the target node set.
Optionally, the executing the query task by using a node in the target node set includes:
utilizing Coordinator nodes in the presto cluster to analyze SQL sentences of the query tasks to obtain a plurality of execution objects;
and scheduling the nodes in the target node set to execute each execution object by using a preset scheduling algorithm.
Optionally, the performing SQL statement analysis on the query task by using a Coordinator node in the presto cluster to obtain a plurality of execution objects includes:
calling SqlParser to analyze the query task by using the Coordinator node to obtain a State object;
and packaging the State element object into a QueryStarter object to obtain the execution object.
In order to solve the above problem, the present invention further provides a node scheduling apparatus, including:
an information screening module, configured to obtain a node plan information set, where each node plan information in the node plan information set includes: node IP, effective time interval, node grouping information and node preset state; acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
the node marking module is used for dynamically acquiring all running nodes and corresponding running node IPs in the presto cluster, performing IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking a grouping label on each screened node by using the node grouping information to obtain a first node set;
the node scheduling module is used for identifying the node state of each node in the first node set and screening the nodes of the first node set according to the node state identification result to obtain a second node set; and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
In order to solve the above problem, the present invention also provides an electronic device, including:
a memory storing at least one computer program; and
and the processor executes the computer program stored in the memory to realize the node scheduling method.
In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is executed by a processor in an electronic device to implement the node scheduling method described above.
In the embodiment of the present invention, the nodes in the second node set are selected to execute the query tasks according to the user service tags and the group tags, and the corresponding nodes are scheduled to execute the query tasks according to the difference in the user service tags corresponding to each query task, so that the query tasks of different user service groups can run independently on different nodes without mutual influence, and the node scheduling is more flexible.
Drawings
Fig. 1 is a schematic flowchart of a node scheduling method according to an embodiment of the present invention;
fig. 2 is a schematic block diagram of a node scheduling apparatus according to an embodiment of the present invention;
fig. 3 is a schematic internal structural diagram of an electronic device implementing a node scheduling method according to an embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the invention provides a node scheduling method. The executing body of the node scheduling method includes, but is not limited to, at least one of electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiments of the present application. In other words, the node scheduling method may be performed by software or hardware installed in the terminal device or the server device, and the software may be a block chain platform. The server includes but is not limited to: the cloud server can be an independent server, or can be a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, Network service, cloud communication, middleware service, domain name service, security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.
Referring to fig. 1, which is a schematic flow chart of a node scheduling method according to an embodiment of the present invention, in an embodiment of the present invention, the node scheduling method includes:
s1, acquiring a node plan information set, wherein each node plan information in the node plan information set includes: node IP, effective time interval, node grouping information and node preset state;
in detail, in the embodiment of the present invention, the node plan information set is an effective time interval, node grouping information, and a node preset state set for each node in the presto cluster. Wherein, the node IP is the IP address of the node; the effective time interval is the effective time interval of the node; the node preset state is a manually set node state, and comprises: active and non-active states, such as: the effective time interval is [8:00,9:00], the preset state is alive, and then the effective time of the node is 8:00-9: 00; the node grouping information is a corresponding service grouping set by a node, the state is alive, each node can only correspond to one service grouping, and the node can only execute the query task of the service corresponding to the service grouping, such as: and the service group corresponding to the node is service A, so that the node can only execute the query task corresponding to the service A.
In another embodiment of the present invention, the node planning information set may be stored in a blockchain node, and the data access efficiency is improved by using the characteristic of high throughput of the blockchain node.
S2, acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
in the embodiment of the present invention, the task effective time interval is a time interval for executing a corresponding service query task in a plan, and further, in order to screen a node whose preset state is active and which can perform service query in an effective time interval, the embodiment of the present invention screens the node plan information set according to the effective time interval and the preset state.
According to one embodiment of the invention, the node plan information set is screened according to the task effective time interval, the node preset state and the node plan information set.
In detail, in the embodiment of the present invention, the screening the node plan information set according to the valid time interval, the preset node state, and the task valid time interval to obtain a target node plan information set includes:
extracting node plan information of which the preset state of the node is an alive state in the node plan information set to obtain an initial node plan information set;
and screening the node plan information of the effective time interval in the task effective time interval in the initial node plan information set to obtain a target node plan information set.
S3, dynamically acquiring all running nodes and corresponding running node IPs in the presto cluster, carrying out IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking packet labels on each screened node by using the node packet information to obtain a first node set;
in detail, the presto cluster in the embodiment of the present invention is a running presto server cluster.
In the embodiment of the invention, the restful interface of a discovery server (discovery server) is utilized to dynamically acquire the IP of all running nodes in a presto cluster to obtain the corresponding running node IP, IP matching screening is carried out according to each node IP in the target node plan information set and the dynamically acquired running node IP, and the nodes corresponding to the running node IP which is the same as the node IP are screened.
Further, in order to determine which service-corresponding query tasks can be executed by each screened node, the embodiment of the present invention obtains node grouping information corresponding to the node IP that is the same as the operating node IP in the target node plan information set, and obtains target node grouping information; grouping label marking is carried out on the corresponding nodes by utilizing the target node grouping information; and summarizing all the nodes marked by the labels to obtain the first node set, so that the screened nodes have corresponding association relation with corresponding node grouping information, the screened nodes are conveniently grouped and scheduled according to services, all the nodes marked by the labels are summarized to obtain the first node set, at this time, each node in the first node set already contains related grouping information, and the nodes in the first node set can be grouped and scheduled according to services in subsequent steps.
S4, identifying the node state of each node in the first node set, and screening the first node set according to the result of the node state identification to obtain a second node set;
in one embodiment of the present invention, a node state of each node in the first node set is obtained; and screening the nodes with the node states of active in the first node set to obtain the second node set.
In detail, in the embodiment of the present invention, a discovery node manager class is used to obtain a node _ version of each node in the first node set (the node _ version is automatically generated when each node is started), so as to obtain a node state of each node, and further, a node whose node _ version is "active" is screened out, so as to obtain the second node set.
In the embodiment of the present invention, the screening process of the second node set does not affect the grouping labels corresponding to the nodes, so that each node in the second node set has a corresponding grouping label.
And S5, when a query task request is acquired within the task effective time interval, responding to the query task request, extracting a query task and a corresponding user service label in the query task request, and selecting a node in the second node set according to the user service label and the grouping label to execute the query task.
In the embodiment of the invention, the query task request received only in the task effective time interval can be responded, and the query task requests received in other time intervals are not responded.
In detail, in the embodiment of the present invention, the query task is a query task of different user services, the service tag is a service to which the query task belongs, and since there are many types of user services, in order to avoid that query tasks of different services are divided into the same nodes to affect each other, in the embodiment of the present invention, in order to more flexibly implement isolation of physical resources by using different nodes, a node whose packet tag corresponds to the second node set is screened as a node of the user service tag to obtain a target node set, and the node in the target node set is used to execute the query task, for example: and if the user service label is service A, screening all nodes with grouping labels as service A in the second node set to obtain the target node set.
Specifically, the query task in the embodiment of the present invention is an SQL query statement.
Further, in the embodiment of the present invention, executing the query task by using the nodes in the target node set includes: and utilizing Coordinator nodes in the presto cluster to carry out SQL statement analysis on the query task to obtain a plurality of execution objects, and utilizing a preset scheduling algorithm to schedule the nodes in the target node set to execute each execution object. Optionally, the scheduling algorithm may be a topological scheduling algorithm.
In detail, the Coordinator node is used for calling SqlParser to analyze the query task to obtain a State object; and packaging the State element object into a QueryStarter object to obtain the execution object.
Fig. 2 is a functional block diagram of the node scheduling apparatus according to the present invention.
The node scheduling apparatus 100 according to the present invention may be installed in an electronic device. According to the implemented functions, the node scheduling apparatus may include an information screening module 101, a node marking module 102, and a node scheduling module 103, which may also be referred to as a unit, and refers to a series of computer program segments that can be executed by a processor of an electronic device and can perform fixed functions, and are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the information screening module 101 is configured to obtain a node plan information set, where each node plan information in the node plan information set includes: node IP, effective time interval, node grouping information and node preset state; acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
the node marking module 102 is configured to dynamically obtain all nodes in a presto cluster and corresponding operating node IPs, perform IP matching screening on all the nodes according to each node IP in the target node plan information set and the operating node IP, and mark a packet label on each screened node by using the node packet information to obtain a first node set;
the node scheduling module 103 is configured to perform node state identification on each node in the first node set, and perform node screening on the first node set according to a result of the node state identification to obtain a second node set; and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
In detail, in the embodiment of the present invention, when the modules in the node scheduling apparatus 100 are used, the same technical means as the node scheduling method described in fig. 1 are used, and the same technical effect can be produced, which is not described herein again.
Fig. 2 is a schematic structural diagram of an electronic device implementing the node scheduling method according to the present invention.
The electronic device may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program, such as a node dispatcher, stored in the memory 11 and operable on the processor 10.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only to store application software installed in the electronic device and various types of data, such as codes of a node scheduler, etc., but also to temporarily store data that has been output or will be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device by running or executing programs or modules (e.g., node schedulers, etc.) stored in the memory 11 and calling data stored in the memory 11.
The communication bus 12 may be a PerIPheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The communication bus 12 is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
Fig. 2 shows only an electronic device with components, and it will be understood by those skilled in the art that the structure shown in fig. 2 does not constitute a limitation of the electronic device, and may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
For example, although not shown, the electronic device may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management and the like are realized through the power management device. The power source may also include any component of one or more dc or ac power sources, recharging devices, power failure classification circuits, power converters or inverters, power status indicators, and the like. The electronic device may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Optionally, the communication interface 13 may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices.
Optionally, the communication interface 13 may further include a user interface, which may be a Display (Display), an input unit (such as a Keyboard (Keyboard)), and optionally, a standard wired interface, or a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The node scheduler stored in the memory 11 of the electronic device is a combination of computer programs, which when executed in the processor 10, can implement:
acquiring a node plan information set, wherein each node plan information in the node plan information set comprises: node IP, effective time interval, node grouping information and node preset state;
acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
dynamically acquiring all running nodes and corresponding running node IPs in a presto cluster, carrying out IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking grouping labels on each screened node by using the node grouping information to obtain a first node set;
identifying the node state of each node in the first node set, and screening the nodes of the first node set according to the result of the node state identification to obtain a second node set;
and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
Specifically, the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the computer program, which is not described herein again.
Further, the electronic device integrated module/unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. The computer readable medium may be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
Embodiments of the present invention may also provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program may implement:
acquiring a node plan information set, wherein each node plan information in the node plan information set comprises: node IP, effective time interval, node grouping information and node preset state;
acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
dynamically acquiring all running nodes and corresponding running node IPs in a presto cluster, carrying out IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking grouping labels on each screened node by using the node grouping information to obtain a first node set;
identifying the node state of each node in the first node set, and screening the nodes of the first node set according to the result of the node state identification to obtain a second node set;
and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
Further, the computer usable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A method for scheduling nodes, the method comprising:
acquiring a node plan information set, wherein each node plan information in the node plan information set comprises: node IP, effective time interval, node grouping information and node preset state;
acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
dynamically acquiring all running nodes and corresponding running node IPs in a presto cluster, carrying out IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking grouping labels on each screened node by using the node grouping information to obtain a first node set;
identifying the node state of each node in the first node set, and screening the nodes of the first node set according to the result of the node state identification to obtain a second node set;
and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
2. The node scheduling method of claim 1, wherein the screening the node plan information set according to the valid time interval, the preset state of the node, and the task valid time interval to obtain a target node plan information set comprises:
extracting node plan information of which the node preset state is an ai ve state in the node plan information set to obtain an initial node plan information set;
and screening the node plan information of the effective time interval in the task effective time interval in the initial node plan information set to obtain a target node plan information set.
3. The node scheduling method of claim 1, wherein the performing IP matching screening on all the nodes according to each node IP in the target node plan information set and the operating node IP, and performing packet tagging on each screened node by using the node packet information to obtain a first node set comprises: screening nodes corresponding to the operating node IP which is the same as the node IP;
selecting node grouping information corresponding to the node IP which is the same as the running node IP in the target node plan information set to obtain target node grouping information;
grouping label marking is carried out on the corresponding nodes by utilizing the target node grouping information;
and summarizing all the nodes marked by the labels to obtain the first node set.
4. The node scheduling method of claim 1, wherein the identifying a node state of each node in the first node set, and performing node screening on the first node set according to a result of the identifying the node state to obtain a second node set, comprises:
acquiring the node state of each node in the first node set;
and screening the nodes with the node states of active in the first node set to obtain the second node set.
5. The node scheduling method according to any one of claims 1 to 4, wherein the selecting a node in the second node set according to the user service label and the packet label to perform a query task comprises:
screening the corresponding grouping labels in the second node set as the nodes of the user service labels to obtain a target node set;
and executing the query task by utilizing the nodes in the target node set.
6. The node scheduling method of claim 5, wherein said performing the query task with a node in the target node set comprises:
utilizing Coordinator nodes in the presto cluster to analyze SQL sentences of the query tasks to obtain a plurality of execution objects;
and scheduling the nodes in the target node set to execute each execution object by using a preset scheduling algorithm.
7. The node scheduling method of claim 6, wherein said performing SQL statement parsing on the query task by using a Coordinator node in the presto cluster to obtain a plurality of execution objects comprises:
calling SqlParser to analyze the query task by using the Coordinator node to obtain a State object;
and packaging the State element object into a QueryStarter object to obtain the execution object.
8. A node scheduling apparatus, comprising:
an information screening module, configured to obtain a node plan information set, where each node plan information in the node plan information set includes: node IP, effective time interval, node grouping information and node preset state; acquiring a task effective time interval, and screening the node plan information set according to the effective time interval, the node preset state and the task effective time interval to obtain a target node plan information set;
the node marking module is used for dynamically acquiring all running nodes and corresponding running node IPs in the presto cluster, performing IP matching screening on all the nodes according to each node IP in the target node plan information set and the running node IPs, and marking a grouping label on each screened node by using the node grouping information to obtain a first node set;
the node scheduling module is used for identifying the node state of each node in the first node set and screening the nodes of the first node set according to the node state identification result to obtain a second node set; and when the query task request is acquired within the task effective time interval, responding to the query task request, extracting the query task and the corresponding user service label in the query task request, and selecting the nodes in the second node set according to the user service label and the grouping label to execute the query task.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor;
wherein the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the node scheduling method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the node scheduling method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111270895.7A CN113918305A (en) | 2021-10-29 | 2021-10-29 | Node scheduling method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111270895.7A CN113918305A (en) | 2021-10-29 | 2021-10-29 | Node scheduling method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113918305A true CN113918305A (en) | 2022-01-11 |
Family
ID=79243561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111270895.7A Pending CN113918305A (en) | 2021-10-29 | 2021-10-29 | Node scheduling method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113918305A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115022157A (en) * | 2022-08-09 | 2022-09-06 | 深圳竹云科技股份有限公司 | Method, device, equipment and medium for node failover in cluster |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150199214A1 (en) * | 2014-01-13 | 2015-07-16 | Electronics And Telecommunications Research Institute | System for distributed processing of stream data and method thereof |
| CN109033123A (en) * | 2018-05-31 | 2018-12-18 | 康键信息技术（深圳）有限公司 | Querying method, device, computer equipment and storage medium based on big data |
| US20190324797A1 (en) * | 2018-04-18 | 2019-10-24 | Alibaba Group Holding Limited | Task processing method, apparatus, and system |
| CN111125178A (en) * | 2018-10-30 | 2020-05-08 | 亿度慧达教育科技（北京）有限公司 | Data query method, device, terminal, presto query engine and storage medium |
| CN111897911A (en) * | 2020-06-11 | 2020-11-06 | 中国科学院计算机网络信息中心 | Unstructured data query method and system based on secondary attribute graph |
| CN111984426A (en) * | 2020-10-09 | 2020-11-24 | 中国平安人寿保险股份有限公司 | Task scheduling method and device, electronic equipment and storage medium |
-
2021
- 2021-10-29 CN CN202111270895.7A patent/CN113918305A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150199214A1 (en) * | 2014-01-13 | 2015-07-16 | Electronics And Telecommunications Research Institute | System for distributed processing of stream data and method thereof |
| US20190324797A1 (en) * | 2018-04-18 | 2019-10-24 | Alibaba Group Holding Limited | Task processing method, apparatus, and system |
| CN109033123A (en) * | 2018-05-31 | 2018-12-18 | 康键信息技术（深圳）有限公司 | Querying method, device, computer equipment and storage medium based on big data |
| CN111125178A (en) * | 2018-10-30 | 2020-05-08 | 亿度慧达教育科技（北京）有限公司 | Data query method, device, terminal, presto query engine and storage medium |
| CN111897911A (en) * | 2020-06-11 | 2020-11-06 | 中国科学院计算机网络信息中心 | Unstructured data query method and system based on secondary attribute graph |
| CN111984426A (en) * | 2020-10-09 | 2020-11-24 | 中国平安人寿保险股份有限公司 | Task scheduling method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 刘瀚文;姜春兰;李明;程鑫轶;: "网络化弹药的临时集中控制轮询时分多址协议", 探测与控制学报, no. 02, 26 April 2013 (2013-04-26) * |
| 朱洁;李雯睿;王江平;赵红;: "基于节点集计算能力差异的Hadoop自适应任务调度算法", 计算机应用, no. 04, 10 April 2016 (2016-04-10) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115022157A (en) * | 2022-08-09 | 2022-09-06 | 深圳竹云科技股份有限公司 | Method, device, equipment and medium for node failover in cluster |
| CN115022157B (en) * | 2022-08-09 | 2022-11-15 | 深圳竹云科技股份有限公司 | Method, device, equipment and medium for node failover in cluster |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113342472A (en) | Micro-service cluster creating method and device, electronic equipment and readable storage medium | |
| CN111538573A (en) | Asynchronous task processing method and device and computer readable storage medium | |
| CN113806434A (en) | Big data processing method, device, equipment and medium | |
| CN112256783A (en) | Data export method and device, electronic equipment and storage medium | |
| CN113890712A (en) | Data transmission method and device, electronic equipment and readable storage medium | |
| CN113868528A (en) | Information recommendation method and device, electronic equipment and readable storage medium | |
| CN115129753A (en) | Data blood relationship analysis method and device, electronic equipment and storage medium | |
| CN114880368A (en) | Data query method and device, electronic equipment and readable storage medium | |
| CN113918305A (en) | Node scheduling method and device, electronic equipment and readable storage medium | |
| CN115314570B (en) | Data issuing method, device, equipment and medium based on protocol development framework | |
| CN114625512A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| CN114817408A (en) | Scheduling resource identification method and device, electronic equipment and storage medium | |
| CN115220740A (en) | Database environment deployment method and device, electronic equipment and storage medium | |
| CN113918296A (en) | Model training task scheduling execution method and device, electronic equipment and storage medium | |
| CN115145870A (en) | Method and device for positioning reason of failed task, electronic equipment and storage medium | |
| CN114356530A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| CN114625712A (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN114675976A (en) | GPU sharing method, device, equipment and medium based on kubernets | |
| CN114626103A (en) | Data consistency comparison method, device, equipment and medium | |
| CN114547011A (en) | Data extraction method and device, electronic equipment and storage medium | |
| CN112527443A (en) | Prompt box display method and device, electronic equipment and computer readable storage medium | |
| CN112328656A (en) | Service query method, device, equipment and storage medium based on middle platform architecture | |
| CN115174698B (en) | Market data decoding method, device, equipment and medium based on table entry index | |
| CN111612417A (en) | System resource allocation method and device, electronic equipment and storage medium | |
| CN114253525A (en) | Lightweight component bus implementation method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |