CN115361331B - Channel selection method, system, device, electronic equipment and storage medium - Google Patents

Channel selection method, system, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115361331B
CN115361331B CN202210887128.9A CN202210887128A CN115361331B CN 115361331 B CN115361331 B CN 115361331B CN 202210887128 A CN202210887128 A CN 202210887128A CN 115361331 B CN115361331 B CN 115361331B
Authority
CN
China
Prior art keywords
routing
channel
node
channels
working nodes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210887128.9A
Other languages
Chinese (zh)
Other versions
CN115361331A (en
Inventor
袁玮玮
吴桐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN202210887128.9A priority Critical patent/CN115361331B/en
Publication of CN115361331A publication Critical patent/CN115361331A/en
Application granted granted Critical
Publication of CN115361331B publication Critical patent/CN115361331B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H04L45/121—Shortest path evaluation by minimising delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H04L45/124—Shortest path evaluation using a combination of metrics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H04L45/125—Shortest path evaluation based on throughput or bandwidth
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/32—Flooding
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
    • Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The application relates to a channel selection method, a system, a device, electronic equipment and a storage medium, which are applied to the technical field of data processing, wherein the method comprises the following steps: obtaining channel operation data of channels among working nodes in a workflow, wherein the channels among the working nodes comprise at least two channels; determining a target channel with the minimum routing cost among the working nodes based on the channel operation data; and transmitting and processing the task to be processed among the working nodes according to the target channel. The method and the device solve the problems that in the prior art, channel tasks are unevenly distributed and system breakdown is easy to occur.

Description

Channel selection method, system, device, electronic equipment and storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a channel selection method, system, device, electronic apparatus, and storage medium.
Background
The pipeline (workflow) is a common tool in the development of internet products, a plurality of channels are usually arranged between two adjacent nodes of the workflow, when data is transmitted between the nodes through the channels, a random mode is usually adopted to transmit the data from one of the channels, so that certain channels are congested, other channels are idle, channel tasks are unevenly distributed, and system breakdown is easy to occur.
Disclosure of Invention
The application provides a channel selection method, a system, a device, electronic equipment and a storage medium, which are used for solving the problem that in the prior art, channel tasks are unevenly distributed and system breakdown is easy to cause.
In a first aspect, an embodiment of the present application provides a channel selection method, including:
Obtaining channel operation data of channels among working nodes in a workflow, wherein the channels among the working nodes comprise at least two channels;
Determining a target channel with the minimum routing cost among the working nodes based on the channel operation data;
and transmitting and processing the task to be processed among the working nodes according to the target channel.
Optionally, the determining, based on the channel operation data, the target channel with the minimum routing cost between the working nodes includes:
determining routing information among the working nodes based on the channel operation data, wherein the routing information comprises the number of routing channels and routing parameters required by the routing channels;
the target channel is determined based on the number of routing channels and the routing parameters.
Optionally, the determining, based on the channel operation data, routing information between the working nodes includes:
Controlling a first node in the workflow to send flooding broadcast information to a second node;
Determining a routing parameter of the second node receiving the flooding broadcast information from each channel based on the channel operation data;
determining the channel from the first node to the second node as a routing channel;
and determining the routing parameters and the routing channels as the routing information.
Optionally, the first node and the second node are two adjacent working nodes in the workflow;
The controlling the first node in the workflow to send the flooding broadcast information to the second node includes:
Controlling each first node in the workflow to simultaneously send the flooding broadcast information to the second nodes adjacent to the first node; or alternatively, the first and second heat exchangers may be,
The first node is a starting node, the second node is a terminating node, and the workflow also comprises an intermediate node;
The controlling the first node in the workflow to send the flooding broadcast information to the second node includes:
And controlling the first node to send the flooding broadcast information to the second node through the intermediate node until the second node receives the flooding broadcast information.
Optionally, the channel operation data includes a channel execution time length and a channel queuing time length; the determining, based on the channel operation data, a routing parameter of the second node that receives the flooding broadcast information from each channel includes:
Determining the channel queuing time length as the routing time length of the second node receiving the flooding broadcasting information from each channel;
Determining average flow data among the working nodes according to the channel execution time length and the channel queuing time length;
and determining the routing time length and/or the average flow data as the routing parameter.
Optionally, the determining average flow data between the working nodes according to the channel execution duration and the channel queuing duration includes:
determining the average task processing capacity of the channel according to the channel queuing time and the channel execution time;
and determining the average task processing capacity as the average flow data.
Optionally, the routing parameters include routing duration and/or routing average traffic; the determining the target channel based on the number of routing channels and the routing parameters includes:
judging whether the routing parameters meet preset routing conditions or not, wherein the preset routing conditions comprise that the routing duration exceeds a preset duration threshold value and/or the routing average flow is greater than a preset flow threshold value;
If not, determining a routing channel with the shortest routing duration or the smallest routing average flow as the target channel;
If yes, judging whether the number of the routing channels reaches a channel threshold value;
if the number of the routing channels does not reach the channel threshold, creating new routing channels among the working nodes.
In a second aspect, an embodiment of the present application provides a channel selection system, including: a workflow platform and workflow management platform;
the workflow platform is used for sending channel operation data of each channel among the working nodes in the workflow to the workflow control platform, wherein at least two channels are arranged among the working nodes;
The workflow control platform is used for acquiring channel operation data of each channel between working nodes in the workflow; determining a target channel with the minimum routing cost among the working nodes based on the channel operation data; and transmitting and processing the task to be processed among the working nodes according to the target channel.
In a third aspect, an embodiment of the present application provides a channel selection apparatus, including:
The system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring channel operation data of channels among working nodes in a workflow, and the working nodes comprise at least two channels;
The determining module is used for determining a target channel with the minimum routing cost among the working nodes based on the channel operation data;
and the control module is used for transmitting and processing the task to be processed among the working nodes according to the target channel.
In a fourth aspect, an embodiment of the present application provides an electronic device, including: the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
The memory is used for storing a computer program;
the processor is configured to execute the program stored in the memory, and implement the channel selection method according to the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, where the computer program implements the channel selection method according to the first aspect when executed by a processor.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the channel operation data of each channel between the working nodes in the workflow are obtained, and at least two channels are arranged between the working nodes; determining a target channel with the minimum routing cost among the working nodes based on the channel operation data; and transmitting and processing the task to be processed among the working nodes according to the target channel. Therefore, the target channel is determined through the channel operation data, so that the task to be processed can be transmitted and processed between the working nodes through the target channel with the minimum routing cost, the utilization rate of the channel is improved, and the condition of uneven distribution of the channel task is avoided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is an application scenario diagram of a channel selection method according to an embodiment of the present application;
FIG. 2 is a flow chart of a channel selection method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a working node in a workflow according to an embodiment of the present application;
FIG. 4 is a block diagram of a channel selection system according to an embodiment of the present application;
FIG. 5 is a block diagram of a channel selection device according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
According to an embodiment of the present application, a channel selection method is provided. Alternatively, in the embodiment of the present application, the above-described channel selection method may be applied to a hardware environment composed of the terminal 101 and the server 102 as shown in fig. 1. As shown in fig. 1, the server 102 is connected to the terminal 101 through a network, which may be used to provide services (such as video services, application services, etc.) to the terminal or clients installed on the terminal, and a database may be provided on the server or independent of the server, for providing data storage services to the server 102, where the network includes, but is not limited to: the terminal 101 is not limited to a PC, a mobile phone, a tablet computer, or the like.
The channel selection method according to the embodiment of the present application may be performed by the server 102, may be performed by the terminal 101, or may be performed by both the server 102 and the terminal 101. The terminal 101 may perform the channel selection method according to the embodiment of the present application, or may be performed by a client installed thereon.
Taking a terminal to execute a channel selection method according to an embodiment of the present application as an example, fig. 2 is a schematic flow chart of an alternative channel selection method according to an embodiment of the present application, as shown in fig. 2, a flow of the method may include the following steps:
step 201, obtaining channel operation data of channels between working nodes in a workflow, wherein the channels between the working nodes comprise at least two channels.
In some embodiments, completion of any item is often accomplished in a workflow, which typically includes a plurality of nodes, through interaction between the nodes, to complete the relevant processing of the task. For each node in the workflow, at least two channels are usually arranged between the nodes, and task circulation is realized through the channels.
Specifically, the obtaining condition of the channel operation data may be when a new task to be processed is detected in the workflow, or the obtaining condition may be obtained once every preset time interval.
The channel operation data may be, but is not limited to, structure data between working nodes of a workflow, execution time of each channel between working nodes of the workflow, queuing time of each channel of the workflow, and channel attribute information.
The channel attribute information may include, but is not limited to, the number of channels, and the maximum value of channels between the created working nodes.
Step 202, determining a target channel with the minimum routing cost among the working nodes based on the channel operation data.
In some embodiments, after the channel operation data of each channel is obtained, the channel operation data includes a channel execution time length, a queuing time length, and the like, so that the routing information of each channel between the working nodes can be determined based on the channel operation data, and further, the target channel with the minimum routing cost is determined based on the routing information.
In an alternative embodiment, determining a target channel with the minimum routing cost between the working nodes based on channel operation data includes:
determining routing information among the working nodes based on the channel operation data, wherein the routing information comprises the number of routing channels and routing parameters required by the routing channels; the target channel is determined based on the number of routing channels and the routing parameters.
In some embodiments, the channel operation data generally includes the number of channels between nodes, the channel execution duration, the queuing duration, and the like, so that, based on the channel execution duration and the queuing duration, the routing parameters required by each channel and the number of routing channels between working nodes can be determined, and further, the target channel is determined according to the number of routing channels and the routing parameters.
In the working nodes, whether the channel from the initial node to the second node is congested or not has a certain relation with the number of the routing channels and the routing parameters, so that after the number of the routing channels and the routing parameters are obtained, the target channel with the minimum routing cost can be determined based on the number of the routing channels and the routing parameters.
Wherein the routing parameter may be, but is not limited to, a routing duration or routing average traffic.
In an alternative embodiment, the determining routing information between the working nodes based on the channel operation data includes:
Controlling a first node in the workflow to send flooding broadcast information to a second node; determining a routing parameter of the second node receiving the flooding broadcast information from each channel based on the channel operation data; determining the channel from the first node to the second node as a routing channel; and determining the routing parameters and the routing channels as the routing information.
In some embodiments, since at least one path is included between the working nodes, a calculation of the routing cost is required for each path when determining the target path. Based on the method, the starting node among the working nodes can send the flooding broadcast information to the next node adjacent to the working nodes, and the broadcasting information can be routed in each channel in a flooding broadcast information mode, so that the routing parameters of the flooding broadcast information in each channel are determined based on the channel operation data of each channel and are used as the routing information. And by judging whether the next node of the real node is a termination node or not, when the next node is not the termination node, the next node continuously forwards the flooding broadcast information to the next node, and the process is repeated until the flooding broadcast information is sent to the termination node. Therefore, the channel with the minimum routing cost between two adjacent nodes can be determined through the flow of the flooding broadcast information among the nodes, and the channel with the minimum routing cost is further selected for data transmission, so that the execution time of workflow tasks is prolonged.
It can be understood that the foregoing manner of sending the flooding broadcast information may be from the starting node to the next node in turn, until reaching the terminating node technology; or, the flooding broadcast information can be sent to the next working node of the other working nodes except the termination node in the workflow.
The first node and the second node are all working nodes in the workflow.
In an optional embodiment, the channel operation data includes a channel execution duration and a channel queuing duration; the determining, based on the channel operation data, a routing parameter of the second node that receives the flooding broadcast information from each channel includes:
Determining the channel queuing time length as the routing time length of the second node receiving the flooding broadcasting information from each channel; determining average flow data among the working nodes according to the channel execution time length and the channel queuing time length; and determining the routing time length and/or the average flow data as the routing parameter.
In some embodiments, there are various routing parameters, such as the length of time required for the flooding broadcast information to travel from the originating node to the second node between the nodes, and the average traffic data between the working nodes. The routing duration may be a channel queuing duration of each channel.
The channel execution duration may be a duration required for processing a task when the task is transmitted between the working nodes through the channel. The average flow data may be the number of tasks to be processed in the channel.
In an optional embodiment, the determining average traffic data between the working nodes according to the channel execution duration and the channel queuing duration includes:
Determining the average task processing capacity of the channel according to the channel queuing time and the channel execution time; and determining the average task processing capacity as the average flow data.
In some embodiments, the channel queuing time length can represent the total time length of all tasks in the channel, and the channel execution time length represents the single task execution time length, so that based on the channel queuing time length and the channel execution time length, the evaluation task processing capacity of each channel can be determined by dividing the channel queuing time length and the channel execution time length, and evaluation flow data can be obtained.
In an alternative embodiment, the routing parameter includes a routing duration or a routing average traffic; the determining the target channel based on the number of routing channels and the routing parameters includes:
judging whether the routing parameters meet preset routing conditions or not, wherein the preset routing conditions comprise that the routing duration exceeds a preset duration threshold value and/or the routing average flow is greater than a preset flow threshold value; if not, determining a routing channel with the shortest routing duration or the smallest routing average flow as the target channel; if yes, judging whether the number of the routing channels reaches a channel threshold value; if the number of the routing channels does not reach the channel threshold, creating new routing channels among the working nodes.
In some embodiments, the process of determining the target channel through the routing parameter and the routing channel may be to determine whether the routing parameter meets a preset routing condition, and if not, consider that the current channel can carry a task, and select a channel with the shortest routing duration or the smallest comment traffic from the current channel as the target channel. When the routing parameters meet preset routing conditions, the current channel is considered to be more congested, whether the number of the routing channels reaches a preset channel threshold value is judged, and if the number of the routing channels does not reach the preset channel threshold value, a new channel is created so that the new channel participates in the next channel selection; and if the number of the channels reaches a preset channel threshold, selecting the routing channel with the shortest routing duration or the smallest routing average flow as the target channel.
It will be appreciated that when the creation speed of the new channel is high, the new channel may be used as the target channel.
And 203, transmitting and processing the task to be processed among the working nodes according to the target channel.
In some embodiments, after determining the target channels between the working nodes, the task to be processed is transmitted according to the target channels, so that the utilization rate of the idle channels in the workflow can be improved.
In a specific embodiment, the channel selection method provided by the application is that a workflow control platform (intelligent control system) is built on a workflow platform in the related art, and channel selection is performed based on the workflow control platform, and referring to fig. 3, the method specifically includes:
the first and the workflow platform synchronize the data of the workflow operation (namely channel operation data) to the intelligent management and control system.
Wherein the running data comprises: topology data of the workflow, execution time of each node (channel) of the workflow, queuing time of each node (channel) of the workflow, and channel attribute (meta) information.
Secondly, through characteristic engineering in the intelligent management and control system, the data of workflow operation is mined to meet the data characteristics required by management and control.
Wherein the data characteristics include: the execution and queuing time length of each node (channel) of the workflow is abstracted into the channel average flow condition (TPS), the queuing condition of each node (channel) TPS of the workflow, and whether each node (channel) TPS of the workflow reaches the threshold value of creating a new channel.
Thirdly, calculating the optimal solution (namely a target channel) of the new task passing through the workflow channel path through a Wardrop management and control algorithm in the intelligent management and control system.
Fourth, whether a new channel needs to be opened or not is calculated through Wardrop management and control algorithm.
The calculation method may be that when a certain task node (channel) TPS reaches a new channel creation threshold and the number of channels does not reach the maximum created channel number, a new channel is created.
And fifthly, calling a workflow platform, and controlling the circulation of the task in the workflow task channel until the task is completed by the results calculated in the third step and the fourth step.
Further, based on Wardrop equalization principles in combination with workflow topology and workflow operation data sent to the workflow intelligent management and control system, selecting the current optimal channel path calculation flow may include:
First, starting from the s0 node (first working node in the workflow), the next node s1 of the s0 node is selected as the destination node.
Second, the flooding broadcasting information M is transmitted from s0 to s1.
Thirdly, the information M updates the time length of broadcasting reaching the s1 node of each channel to the information M after broadcasting reaching the s1 node through all channels between the s0 and the s1 nodes, and updates the routing table information of the s0 node.
Fourth, it is judged whether s1 is the termination node d0, if yes, the broadcasting is stopped, and if not, the flooding broadcasting information M is transmitted to the next node of s1 node, and the second and third steps are repeatedly performed.
Fifthly, after the termination node d0 is determined, route searching is carried out according to the routing time length with the minimum cost in the routing information table, and the workflow task is operated.
Further, for the route channel searching process, the first to fifth steps of calculation can be repeated according to a certain pulse frequency (for example, every 5 seconds), and each time an optimal workflow running path during the current pulse interval can be calculated, and the workflow is run according to the current path.
Further, if the channel execution time in the routing table reaches the current channel execution time threshold set in the workflow meta information and the number of channels is less than the maximum value, a current node task operation channel is newly built, and the newly built channel participates in the next optimal path calculation.
It will be appreciated that the above-described channel selection calculation process may also be implemented by training a network model. Specifically, the workflow data is collected and automatically marked, training data construction is performed based on workflow operation data, constructed data is provided for an algorithm, wardrop management and control algorithms are selected, a model is trained, algorithm evaluation is performed in actual operation of the effect, and when an evaluation result reaches expectations, the model is put into operation of the actual workflow.
The pipeline (workflow) in the present application may be, but is not limited to,: professional production Content (Professional Generated Content, PGC), expert production Content (Professionally-produced Content, PPC), production transcoding process of photo text, transcoding work of video, and the like.
Illustratively, when the workflow is a transcoding job of video transcoding, referring to fig. 3, the working nodes thereof include an anti-display node, an audio track extraction node, a preview stream transcoding node, a slice transcoding node, and a publishing node. The number of channels between the anti-display node and the audio track extraction node is 3, the number of channels between the audio track extraction node and the preview stream transcoding node is 2, and the number of channels between the preview stream transcoding node and the slice transcoding node is 1; the number of channels between the slice transcoding node and the publishing node is 5 as an example. Different numbers of channels are configured among the working nodes so as to utilize the channels to carry out streaming of video transcoding tasks.
In the practical application process, after a new video transcoding task is uploaded to a workflow, the new video transcoding task is stored to a start node, two channels are arranged between the start node and an anti-display node, and the start node determines a target channel with the minimum routing cost between the two channels based on the running data of each channel. For example, the number of video transcoding tasks to be processed in the first channel is 2, and the queuing time of the channel is 2 minutes; and the number of video transcoding tasks to be processed in the second channel is 3, and the queuing time of the second channel is 1.5 minutes. Therefore, the routing time length (i.e. queuing time length) of the second channel is shorter than that of the first channel, and based on the routing time length, the second channel is selected as a target channel, so that the video transcoding task flow is transferred to the anti-display node.
It will be appreciated that the target channel may be selected based on the average flow rate. The average flow of the first channel is 1 minute for each task execution duration, and the average flow of the second channel is 0.5 minute for each task execution duration, which means that the average flow of the second channel is smaller than that of the first channel, so that the second channel is selected as the target channel.
In another example, there is a third channel between the start node and the anti-display node, where the number of video transcoding tasks to be processed in the channel is 5, the queuing time of the channel is 20 minutes, the average flow of the channel is 4 minutes, if the preset time threshold is 15 minutes, the preset flow threshold is 3 minutes, and it is known that the routing condition of the third channel has exceeded the preset time threshold and the preset flow threshold, at this time, the system determines whether the highest number of channels is 5, based on the above example, there are three channels, and 5 channels are not reached, so a new channel is created, and when the channel is selected next time, the new channel is used as an alternative.
It will be appreciated that if the number of channels between two working nodes has reached a maximum, for example, the number of channels between a piece of transcoding nodes and a distribution node is 5, and the maximum is reached, then, when selecting the target channel, the channel with the shortest routing duration or the smallest average traffic is selected as the target channel.
Based on the channel selection method, the target channel is determined through the channel operation data, so that the task to be processed can be transmitted and processed between the working nodes through the target channel with the minimum routing cost, the utilization rate of the channel is improved, and the situation of uneven distribution of the channel tasks is avoided.
At present, tens of millions of channels run on line, so that various business of a content center station is served, a pipeline (workflow) is smooth and normally operated, and the on-line timeliness of various content products of a company can be greatly improved.
The application can further optimize the algorithm effect by using Wardrop in the pipeline (workflow), for example, the transcoding work of the video, a large amount of data can be generated in the transcoding process, and the data are further analyzed and sorted to find out the characteristic parameters which are more suitable for the algorithm. Providing an optimal solution for overall task completion. Therefore, the smooth and normal operation of the assembly line (workflow) can greatly improve the on-line timeliness of each content product, and good guarantee is provided for the preemptive duration of the content service.
Based on the same concept, the embodiment of the present application provides a channel selection system, and the specific implementation of the system may be referred to the description of the embodiment of the method, and the details are not repeated, as shown in fig. 4, where the system mainly includes: a workflow platform and workflow management platform;
The workflow platform 401 is configured to send channel operation data of channels between working nodes in a workflow to the workflow control platform, where the working nodes include at least two channels;
The workflow control platform 402 is configured to obtain channel operation data of channels between working nodes in a workflow; determining a target channel with the minimum routing cost among the working nodes based on the channel operation data; and transmitting and processing the task to be processed among the working nodes according to the target channel.
Based on the same concept, the embodiment of the present application provides a channel selection device, and the specific implementation of the device may be referred to the description of the embodiment of the method, and the repetition is omitted, as shown in fig. 5, where the device mainly includes:
An obtaining module 501, configured to obtain channel operation data of channels between working nodes in a workflow, where the working nodes include at least two channels;
a determining module 502, configured to determine, based on the channel operation data, a target channel with a minimum routing cost between the working nodes;
And the control module 503 is used for transmitting and processing the task to be processed among the working nodes according to the target channel.
Based on the same concept, the embodiment of the application also provides an electronic device, as shown in fig. 6, where the electronic device mainly includes: processor 601, memory 602, and communication bus 603, wherein processor 601 and memory 602 accomplish communication with each other through communication bus 603. The memory 602 stores a program executable by the processor 601, and the processor 601 executes the program stored in the memory 602 to implement the following steps:
Obtaining channel operation data of channels among working nodes in a workflow, wherein the channels among the working nodes comprise at least two channels;
Determining a target channel with the minimum routing cost among the working nodes based on the channel operation data;
and transmitting and processing the task to be processed among the working nodes according to the target channel.
The communication bus 603 mentioned in the above electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, or the like. The communication bus 603 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus.
The memory 602 may include random access memory (Random Access Memory, RAM) or non-volatile memory (nonvolatile memory), such as at least two disk memories. Alternatively, the memory may be at least one memory device located remotely from the aforementioned processor 601.
The processor 601 may be a general-purpose processor, including a central Processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), a digital signal processor (DIGITAL SIGNAL Processing, DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware components.
In a further embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to perform the channel selection method described in the above embodiment.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, by a wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, microwave, etc.) means from one website, computer, server, or data center to another. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape, etc.), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (9)

1. A channel selection method, comprising:
Obtaining channel operation data of channels among working nodes in a workflow, wherein the channels among the working nodes comprise at least two channels;
Determining a target channel with the minimum routing cost among the working nodes based on the channel operation data;
transmitting and processing tasks to be processed among the working nodes according to the target channel;
the determining, based on the channel operation data, a target channel with the minimum routing cost between the working nodes includes:
Determining routing information among the working nodes based on the channel operation data, wherein the routing information comprises the number of routing channels and routing parameters required by the routing channels; the routing parameters comprise routing duration or routing average traffic;
judging whether the routing parameters meet preset routing conditions or not, wherein the preset routing conditions comprise that the routing duration exceeds a preset duration threshold value and/or the routing average flow is greater than a preset flow threshold value;
If not, determining a routing channel with the shortest routing duration or the smallest routing average flow as the target channel;
If yes, judging whether the number of the routing channels reaches a channel threshold value;
if the number of the routing channels does not reach the channel threshold, creating new routing channels among the working nodes.
2. The channel selection method according to claim 1, wherein the determining routing information between the working nodes based on the channel operation data includes:
controlling a first node in the workflow to send flooding broadcast information to a second node, wherein the first node and the second node are working nodes in the workflow;
Determining a routing parameter of the second node receiving the flooding broadcast information from each channel based on the channel operation data;
determining the channel from the first node to the second node as a routing channel;
and determining the routing parameters and the routing channels as the routing information.
3. The channel selection method according to claim 2, wherein the first node and the second node are two adjacent working nodes in the workflow;
The controlling the first node in the workflow to send the flooding broadcast information to the second node includes:
Controlling each first node in the workflow to simultaneously send the flooding broadcast information to the second nodes adjacent to the first node; or alternatively, the first and second heat exchangers may be,
The first node is a starting node, the second node is a terminating node, and the workflow also comprises an intermediate node;
The controlling the first node in the workflow to send the flooding broadcast information to the second node includes:
And controlling the first node to send the flooding broadcast information to the second node through the intermediate node until the second node receives the flooding broadcast information.
4. The channel selection method according to claim 2, wherein the channel operation data includes a channel execution duration and a channel queuing duration; the determining, based on the channel operation data, a routing parameter of the second node that receives the flooding broadcast information from each channel includes:
Determining the channel queuing time length as the routing time length of the second node receiving the flooding broadcasting information from each channel;
Determining average flow data among the working nodes according to the channel execution time length and the channel queuing time length;
and determining the routing time length and/or the average flow data as the routing parameter.
5. The method of claim 4, wherein determining average traffic data between the working nodes according to the channel execution duration and the channel queuing duration comprises:
determining the average task processing capacity of the channel according to the channel queuing time and the channel execution time;
and determining the average task processing capacity as the average flow data.
6. A channel selection system, comprising: a workflow platform and workflow management platform;
the workflow platform is used for sending channel operation data of each channel among the working nodes in the workflow to the workflow control platform, wherein at least two channels are arranged among the working nodes;
The workflow control platform is used for acquiring channel operation data of each channel between working nodes in the workflow; determining a target channel with the minimum routing cost among the working nodes based on the channel operation data; transmitting and processing tasks to be processed among the working nodes according to the target channel;
the determining, based on the channel operation data, a target channel with the minimum routing cost between the working nodes includes:
Determining routing information among the working nodes based on the channel operation data, wherein the routing information comprises the number of routing channels and routing parameters required by the routing channels; the routing parameters comprise routing duration or routing average traffic;
judging whether the routing parameters meet preset routing conditions or not, wherein the preset routing conditions comprise that the routing duration exceeds a preset duration threshold value and/or the routing average flow is greater than a preset flow threshold value;
If not, determining a routing channel with the shortest routing duration or the smallest routing average flow as the target channel;
If yes, judging whether the number of the routing channels reaches a channel threshold value;
if the number of the routing channels does not reach the channel threshold, creating new routing channels among the working nodes.
7. A channel selection device, comprising:
The system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring channel operation data of channels among working nodes in a workflow, and the working nodes comprise at least two channels;
The determining module is used for determining a target channel with the minimum routing cost among the working nodes based on the channel operation data;
The control module is used for transmitting and processing the task to be processed among the working nodes according to the target channel;
the determining, based on the channel operation data, a target channel with the minimum routing cost between the working nodes includes:
Determining routing information among the working nodes based on the channel operation data, wherein the routing information comprises the number of routing channels and routing parameters required by the routing channels; the routing parameters comprise routing duration or routing average traffic;
judging whether the routing parameters meet preset routing conditions or not, wherein the preset routing conditions comprise that the routing duration exceeds a preset duration threshold value and/or the routing average flow is greater than a preset flow threshold value;
If not, determining a routing channel with the shortest routing duration or the smallest routing average flow as the target channel;
If yes, judging whether the number of the routing channels reaches a channel threshold value;
if the number of the routing channels does not reach the channel threshold, creating new routing channels among the working nodes.
8. An electronic device, comprising: the device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
The memory is used for storing a computer program;
The processor is configured to execute a program stored in the memory, and implement the channel selection method according to any one of claims 1 to 5.
9. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the channel selection method of any one of claims 1-5.
CN202210887128.9A 2022-07-26 2022-07-26 Channel selection method, system, device, electronic equipment and storage medium Active CN115361331B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210887128.9A CN115361331B (en) 2022-07-26 2022-07-26 Channel selection method, system, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210887128.9A CN115361331B (en) 2022-07-26 2022-07-26 Channel selection method, system, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115361331A CN115361331A (en) 2022-11-18
CN115361331B true CN115361331B (en) 2024-06-14

Family

ID=84031941

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210887128.9A Active CN115361331B (en) 2022-07-26 2022-07-26 Channel selection method, system, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115361331B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101047660A (en) * 2006-03-28 2007-10-03 朗迅科技公司 Method for transmitting data

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20050040915A (en) * 2005-02-04 2005-05-03 해리스 코포레이션 Multi-channel mobile ad hoc network
CN101378363B (en) * 2008-09-28 2011-07-13 中兴通讯股份有限公司 Method for selecting channel and apparatus thereof
CN101686179B (en) * 2008-09-28 2013-01-09 华为技术有限公司 Transmission method, nodes and point-to-point network of link status release information
WO2019241992A1 (en) * 2018-06-22 2019-12-26 华为技术有限公司 Routing method, node and computer storage medium
CN114286413B (en) * 2021-11-02 2023-09-19 北京邮电大学 TSN network joint routing and stream distribution method and related equipment
CN114281573A (en) * 2021-12-28 2022-04-05 城云科技（中国）有限公司 Workflow data interaction method and device, electronic device and readable storage medium
CN114598654B (en) * 2022-01-30 2024-07-05 阿里巴巴（中国）有限公司 Flow equalization processing method and device based on content delivery network CDN

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101047660A (en) * 2006-03-28 2007-10-03 朗迅科技公司 Method for transmitting data

Also Published As

Publication number Publication date
CN115361331A (en) 2022-11-18

Similar Documents

Publication Publication Date Title
US10664872B2 (en) Systems and methods for generating network intelligence through real-time analytics
US11411865B2 (en) Network resource scheduling method, apparatus, electronic device and storage medium
US20230038780A1 (en) Dynamic Loop Detection and Suppression
CN108959319B (en) Information pushing method and device
US10715638B2 (en) Method and system for server assignment using predicted network metrics
US10374995B2 (en) Method and apparatus for predicting unwanted electronic messages for a user
CN111083050B (en) Data stream transmission method and device based on software defined network
US11979761B2 (en) Method and apparatus for detecting time series data
WO2011008145A1 (en) Providing content by using a social network
CN104486129B (en) The method and system of application service quality are ensured under distributed environment
CN108156257B (en) Information pushing method and device
CN115470867A (en) Agent matching method, device, equipment and storage medium based on knowledge graph
CN109120527B (en) Method and system for transmitting service flow
CN115361331B (en) Channel selection method, system, device, electronic equipment and storage medium
CN111555918B (en) Return source optimization method and device and electronic equipment
CN111405614B (en) Method for calculating APP load sharing at mobile edge
KR20190114126A (en) Software-defined network controller for controlling message transmission to switch based on processing completion delay time of message and method thereof
CN112749204B (en) Method and device for reading data
CN113689222A (en) Block chain-based digital currency transaction network topology data analysis and optimization method and system
CN113706304A (en) Block chain-based digital currency transaction node IP tracing method and system
CN112217740B (en) Request processing method and device based on edge computing network
CN117176839B (en) Remote measurement message transmission method, device, communication equipment and storage medium
CN111327688B (en) Resource scheduling method, server and readable storage medium
CN105357173A (en) Data resource processing method and system, electronic equipment and terminal adopted thereby
WO2024094279A1 (en) A machine learning orchestrator entity for a machine learning system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant