CN117939535A - Dependency task unloading method, terminal and storage medium in V2V scene - Google Patents

Dependency task unloading method, terminal and storage medium in V2V scene Download PDF

Info

Publication number
CN117939535A
CN117939535A CN202410316670.8A CN202410316670A CN117939535A CN 117939535 A CN117939535 A CN 117939535A CN 202410316670 A CN202410316670 A CN 202410316670A CN 117939535 A CN117939535 A CN 117939535A
Authority
CN
China
Prior art keywords
subtask
vehicle
subtasks
task
unloading
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410316670.8A
Other languages
Chinese (zh)
Other versions
CN117939535B (en
Inventor
张本宏
何聪
胡琪炜
徐浩
毕翔
杜朝阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intelligent Manufacturing Institute of Hefei University Technology
Original Assignee
Intelligent Manufacturing Institute of Hefei University Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intelligent Manufacturing Institute of Hefei University Technology filed Critical Intelligent Manufacturing Institute of Hefei University Technology
Priority to CN202410316670.8A priority Critical patent/CN117939535B/en
Publication of CN117939535A publication Critical patent/CN117939535A/en
Application granted granted Critical
Publication of CN117939535B publication Critical patent/CN117939535B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • H04W28/09—Management thereof
    • H04W28/0925—Management thereof using policies
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • H04W28/09—Management thereof
    • H04W28/0958—Management thereof based on metrics or performance parameters
    • H04W28/0967—Quality of Service [QoS] parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • H04W4/46—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for vehicle-to-vehicle communication [V2V]

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Optimization (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Algebra (AREA)
  • Probability & Statistics with Applications (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to the technical field of internet of vehicles (IOT) edge unloading and discloses a dependent task unloading method, a terminal and a storage medium in a V2V scene. The offloading method divides all subtasks in a computing task into critical subtasks and non-critical subtasks. When the key subtasks are unloaded, the key subtasks are duplicated and unloaded to two service vehicles; when the non-critical subtasks are offloaded and the offloading fails, a second offloading is performed on them. And then, a kinematic model, a communication model, a subtask calculation delay model, a subtask execution priority model and a link reliability model between the task vehicle and the service vehicle are built, and a constraint problem for generating an optimal unloading strategy is built. Modeling the constraint problem as a Markov decision process, solving the Markov decision process by adopting a deep reinforcement learning algorithm to obtain an optimal unloading strategy, and unloading the computing task to be unloaded of the task vehicle to the service vehicle according to the optimal unloading strategy. The invention improves the reliability of the task-dependent unloading.

Description

Dependency task unloading method, terminal and storage medium in V2V scene
Technical Field
The invention relates to the technical field of internet of vehicles edge unloading, in particular to a task-dependent unloading method under a V2V scene, a computer terminal applying the method and a computer-readable storage medium.
Background
With the development of 5G and Internet of vehicles, the explosive growth of novel vehicle-mounted applications such as automatic driving, augmented reality, virtual reality and the like brings great convenience to the life of people. At the same time, however, more and more computationally intensive and delay-sensitive computing tasks are being created, but since the computing and storage resources of the vehicle itself are limited, there are insufficient resources to meet the resource requirements of these tasks, which constitutes a significant challenge in ensuring the quality of service required by the vehicle. V2V (Vehicle-to-Vehicle) unloading is considered a very promising approach to solve the above-mentioned problems. For V2V offloading, many scholars have made great progress in optimizing task delays, reducing energy consumption, and stimulating the joining and offloading of vehicles with idle computing resources. However, the influence of the high-speed mobility of the vehicle on the unloading reliability is not considered, and the influence of the vehicle maneuver on the unloading reliability is reduced by partial students in a relatively passive manner.
For example, ahsan et al, document "A novel contract theory-based incentive mechanism for cooperative task-offloading in electricalvehicular networks[J]. IEEE Transactions on Intelligent Transportation Systems,2021, 23(7): 8380-8395."（ discloses a mechanism for electric vehicle network collaborative task offloading excitation based on contract theory [ J ]. IEEE intelligent transportation system theory), which is capable of reducing the influence of high-speed mobility by setting excitation measures to cause a service vehicle and a task vehicle to increase or decrease acceleration of each other synchronously during offloading. For this study, there is a problem in implementation due to uncertainty of various situations in a real scene.
C-V2X movement perception dynamic unloading strategy [ J ]. Physical communication under "Mobility-aware dynamic offloading strategy for C-V2X under multi-access edge computing[J]. Physical Communication, 2021, 49: 101446."（ multi-access edge calculation proposed by Li et al reduces the influence caused by high-speed mobility by establishing evaluation indexes and selecting proper service vehicles. However, this study also has problems in that the currently selected suitable service vehicle may not be the optimal solution due to unpredictability of vehicle travel, and there may be cases where unloading fails.
Waheed et al propose "An infrastructure-assisted job scheduling and task coordination in volunteer computing-based VANET[J]. Complex&Intelligent Systems, 2023,9:3613-3633."（ to divide a task into several subtasks based on infrastructure assisted job scheduling and task coordination J. Complex and intelligent systems in volunteer computing VANET, to offload the subtasks to a vehicle with free computing resources in the same RSU (roadside unit) for computing, to replicate the task and to select one more vehicle for offloading if the subtasks are offloaded to the vehicle for vehicles that may be driving out of the RSU communication range. For the study of Waheed et al, firstly, it does not consider the situation in the RSU-free scenario, and secondly, due to the unpredictable vehicle travel trajectory, it is impossible to determine whether the vehicle will travel out of the communication range for all service vehicles based on the existing vehicle speed and distance from the RSU, and it is possible for all service vehicles to travel out of the communication range.
In summary, the above research focuses on offloading strategies for independent tasks, only for independent tasks, and not for dependent tasks. However, with the development of technology, how to offload a plurality of interdependent subtasks and ensure the reliability of the task offload has become a urgent problem in vehicle network research.
Disclosure of Invention
In order to avoid and overcome the technical problem of lower reliability of dependent task unloading in the existing V2V unloading technology, the invention provides a dependent task unloading method, a terminal and a storage medium in a V2V scene.
In order to achieve the above purpose, the present invention provides the following technical solutions:
The invention discloses a dependent task unloading method in a V2V scene, which comprises the following steps of S1-S3.
S1, taking the urgency degree of a plurality of subtasks with dependency relations in a task vehicle computing task as a dividing standard, and dividing all the subtasks in the computing task into a critical subtask and a non-critical subtask.
When the key subtasks are unloaded, the key subtasks are copied and unloaded to two service vehicles; and when the non-critical subtasks are unloaded and the unloading fails, unloading the subtasks which fail to be unloaded for the second time.
S2, respectively constructing a kinematic model, a communication model, a subtask calculation delay model, a subtask execution priority model and a link reliability model between the task vehicle and the service vehicle, and accordingly establishing a constraint problem for generating an optimal unloading strategy.
And S3, modeling the constraint problem as a Markov decision process, solving the Markov decision process by adopting a deep reinforcement learning algorithm to obtain an optimal unloading strategy, and unloading subtasks to be unloaded of the task vehicle to the service vehicle according to the optimal unloading strategy.
As a further improvement of the above solution, in step S1, the method of dividing all the subtasks in the computing task into critical subtasks and non-critical subtasks includes the following specific steps, namely S11-S13.
S11, calculating the earliest unloading starting time and the latest unloading ending time of each subtask, wherein the calculation formula is as follows:
In the method, in the process of the invention, Representing the ith subtask/>, of the mission vehicle mIs the latest unloading end time of (3); /(I)Representing subtasks/>Is the earliest unload start time in (a); /(I)Representing subtasks/>Is a subsequent sub-task of (a); /(I)Representing subtasks/>Is a preamble sub-task of (2); /(I)Representing subtasks/>Data volume of/>Representing processing subtasks/>Computing resources required for each bit of data; f max represents the most free resources of the service vehicle n; /(I)Representing subtasks/>Is a communication delay of (1); for subtasks/>, without a predecessor subtask，/>Is 0; for subtasks without subsequent subtasks/>，/>To calculate the maximum tolerable delay of the task.
S12, calculating the urgent degree of each subtask, wherein the expression formula is as follows:
In the method, in the process of the invention, Representing subtasks/>Is the urgency of (a); /(I)For subtask/>Is used for the loading of the load onto the vehicle.
S13, comparing the urgency degree of each subtask with a preset threshold value; and when the urgency degree of the subtask is not higher than the preset threshold, defining the subtask as a critical subtask, otherwise defining the subtask as a non-critical subtask.
As a further improvement of the above scheme, subtasksAverage unload time of/>The calculation formula of (2) is as follows:
Wherein n is the maximum number of service vehicles in the service vehicle set; f j is the free computing resources of the j-th service vehicle.
As a further improvement of the above scheme, step S2 includes the following specific steps, namely S21-S26.
S21, constructing a kinematic model; the distance expression formula between the task vehicle and the service vehicle is as follows:
Wherein D m,n (t) represents the distance between the task vehicle m and the service vehicle n at the time t when the sub-task execution phase is completed; d m,n(t0) represents the distance between the task vehicle m and the service vehicle n at the start time t 0 of subtask offloading, D m,n(t0)=xm(t0)－xn(t0),xm(t0) and x n(t0) represent the positions of the task vehicle m and the service vehicle n at t 0, respectively; deltav (t 0)=vm(t0)－vn(t0),vm(t0) and v n(t0) represent the speeds of mission vehicle m and service vehicle n at t 0, respectively; Δa (t 0)=am(t0)－an(t0),am(t0) and a n(t0) represent the acceleration of the mission vehicle m and the service vehicle n at t 0, respectively.
S22, constructing a communication model; wherein the transmission delay of the task vehicle for transmitting the subtasks to the service vehicle through the uploading channel is as followsAnd/>; Wherein R m,n represents the transmission rate of the mission vehicle m to the service vehicle n; representing subtasks/> The size of the data volume transmitted to the service vehicle n is defined by the subtask/>Data volume per se/>And the sum of the data amounts transmitted by the preamble subtasks.
S23, constructing a subtask calculation delay model; wherein the subtasksDelay in servicing vehicle offload tasksAnd/>; Subtask/>The time required to complete the unloading process is/>And (2) and; Subtask/>Unload start time of/>Subtask/>Is (1) the unloading end time isAnd/>，/>; In the/>For subtask/>The unload end time of the preamble subtask.
S24, constructing a subtask execution priority model; wherein subtasks of task vehicle m are definedFor subtask/>Preamble subtasks of/>Execution priority of (3) is higher than/>；/>Is/>And (2) and; Where pri (p) is the subtask/>Execution priority of the preamble subtask.
S25, constructing a link reliability model; wherein, in unloading subtasksWhen the link reliability P m,n,i between the service vehicle n and the mission vehicle m is expressed as follows:
Wherein e is a natural constant; t theroy is the theoretical time for the service vehicle to travel out of the communication range of the task vehicle m at the current speed; the range of values of P m,n,i (0, 1).
S26, establishing a constraint problem as shown below according to the model constructed in the steps S21-S25:
Wherein D success represents the number of calculation tasks successfully unloaded; constraint C1 represents the last subtask completion time of task vehicle m The maximum tolerant time delay T m of the whole calculation task is not exceeded; constraint C2 indicates that initial free resource F n serving vehicle n is between most free resource F max and least free resource F min; constraint C3 represents subtask/>Execution priority/>Lower than the lowest execution priority/>, among all its predecessor subtasks; Constraint C4 represents a subtaskUnloading start time/>Not earlier than the latest unload end time/>, of all its predecessor subtasks; In constraint C5,/>And/>Representing a set of task vehicles and a set of service vehicles, respectively.
As a further improvement of the above scheme, in step S22, the calculation formula of the transmission rate R m,n is:
Wherein B m,n represents the bandwidth of the upload channel; p m denotes the transmission power of the mission vehicle m; Represents the path loss between the service vehicle and the mission vehicle, delta represents the path loss factor, and l represents the distance between the service vehicle and the mission vehicle; h represents the channel fading factor of the uploading link; n 0 represents gaussian white noise power.
As a further improvement of the above solution, in step S3, the markov decision process includes: state space, action space, and reward functions.
The state space is used for representing information parameters of the service vehicle in the communication range of the task vehicle and state parameters of the self-calculation task at the current moment at each moment, and the parameters are used as input states of a deep reinforcement learning algorithm and are represented as s (t ')= [ C (t '), Q (t ') ]; wherein s (t ') is an input state at any time t'; c (t ') is t', said vehicle queue being a member of the service vehicle; and (3) a task queue when Q (t ') is t', wherein the task queue is taken as a member by the subtask.
In the unloading process of a computing task, when one subtask returns a computing result, recalculating the execution priority of the not-scheduled subtask in the computing task; when the service vehicle does not return an unloading result within the expected completion time, judging that the unloading is failed, changing the execution priority of the corresponding subtasks to 0, and putting the subtasks into an unloading queue again to wait for calculation; wherein, for a critical subtask, the maximum value of the predicted completion times offloaded to two service vehicles is taken as the predicted completion time of the critical subtask. When the offloading of the subtask is completed, the execution priority of the subtask becomes 1.
The action space is used for selecting a service vehicle for the subtasks; for a subtask with execution priority of 0, the subtask can be unloaded, so that the subtask can be unloaded to a service vehicle for calculation; all action sets of the action space are represented as action= {1,2, …, n },1,2, …, n being the number of the service vehicle.
The calculation formula of the reward function is; Where r i (t ') represents the task vehicle's scheduling subtasks/>, at tThe awards obtained; t true is subtask/>Is calculated as/>。
As a further improvement of the above, in the vehicle train C (t'), each service vehicle member includes four elements, expressed as:
where x n(t')、an(t')、vn (t') represents the position, acceleration, and speed, respectively, of the service vehicle n at the current time.
In the task queue Q (t'), each subtask member includes eight elements, denoted:
In the method, in the process of the invention, Representing subtasks/>Maximum tolerated time delay of (2); choose denotes a service vehicle to which the subtask is offloaded.
As a further improvement of the above solution, in step S3, the deep reinforcement learning algorithm is DDPG algorithm.
The invention also discloses a computer terminal which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps of the dependent task unloading method under the V2V scene when executing the program.
The invention also discloses a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, implements the steps of the dependent task offloading method in the V2V scenario described above.
Compared with the prior art, the invention has the beneficial effects that:
1. According to the dependency task unloading method under the V2V scene, the concept of a key subtask is provided in consideration of different urgency degrees of each subtask of the dependency task, the subtasks are divided into the key subtask and the non-key subtask before the task is unloaded, and different unloading modes are provided for the subtasks, so that the unloading success rate of the key subtask is ensured as much as possible, the influence of high-speed mobility of a vehicle on the dependent task unloading is reduced, and the unloading success rate of the whole calculation task is improved. On the basis, the invention constructs constraint problems according to the conditions and characteristics of the task vehicles, the service vehicles and the subtask nodes, optimizes the unloading strategy by using a deep reinforcement learning method, and further ensures the unloading reliability of the dependent tasks.
2. The computer terminal and the computer readable storage medium disclosed by the invention can produce the same beneficial effects as the method by applying the task-dependent unloading method, and are not repeated here.
Drawings
FIG. 1 is a flowchart of a method for offloading dependent tasks in a V2V scenario in embodiment 1 of the present invention.
Fig. 2 is a directed acyclic graph of the formation of multiple subtasks in example 1 of the present invention.
Fig. 3 is a system model diagram of a mission vehicle and a service vehicle on a lane in the embodiment 1 of the present invention at the time of V2V communication.
FIG. 4 is a system model diagram of the task vehicle of FIG. 3 in offloading computing tasks to a service vehicle.
Fig. 5 is a bar graph of task success rates for various scenarios under different initial service vehicle numbers in embodiment 1 of the present invention.
Fig. 6 is a histogram of task success rates for various schemes of embodiment 1 of the present invention at different numbers of subtasks.
Fig. 7 is a bar chart of task success rates for various schemes in different communication ranges in embodiment 1 of the present invention.
Fig. 8 is a graph showing average time delay lines for various schemes of example 1 of the present invention for different initial service vehicle numbers.
Fig. 9 is a graph showing the average delay line for various schemes of example 1 according to the present invention at various numbers of subtasks.
Fig. 10 is a graph showing the average delay line for various schemes in example 1 of the present invention under different communication ranges.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
Referring to fig. 1, the present embodiment provides a method for unloading a dependent task in a V2V scenario, which is mainly divided into two parts, wherein the first part is to reduce the influence of high-speed mobility of a vehicle on the unloading of the dependent task, and the second part is to generate an unloading strategy scheme for the dependent task based on the first part scheme. Specifically, the unloading method includes the following steps, S1-S3.
S1, taking the urgency degree of a plurality of subtasks with dependency relations in a task vehicle computing task as a dividing standard, and dividing all the subtasks in the computing task into a critical subtask and a non-critical subtask.
When the key subtasks are unloaded, the key subtasks are copied and unloaded to two service vehicles; and when the non-critical subtasks are unloaded and the unloading fails, unloading the subtasks which fail to be unloaded for the second time.
To address the reliability problem of task offloading, reliability of offloading may be ensured by replicating and offloading subtasks of a computing task to two service vehicles. However, due to the limited number of service vehicles within the communication range of the task vehicle, all subtasks cannot be duplicated and offloaded. Therefore, the invention takes the urgency degree of a plurality of subtasks with dependency relations in the task vehicle computing task as a division standard, divides all the subtasks of the computing task into a key subtask and a non-key subtask, and provides an active replication mechanism based on the key subtask to solve the problem of reliability of task unloading.
The present embodiment classifies vehicles traveling on a road into two categories: mission vehicles and service vehicles. For a set of task vehicles and service vehicles, respectivelyAnd/>To represent. For the service vehicle n, its free computing resources are denoted as F n,Fn∈[Fmin,Fmax. When the service vehicle has idle computing resources, the service vehicle can continuously broadcast own information (including position, speed and idle computing resources), and when the task vehicle needs to unload a computing task, the task vehicle selects a proper service vehicle to unload according to the received service vehicle information and computing task conditions. When the service vehicle receives the calculation task, a message is broadcast to the vehicles within the communication range, which indicates that the service vehicle has no free calculation resources, and when the calculation task is completed, the service vehicle can restart broadcasting the information.
The calculation task of the task vehicle m is denoted as D m=(Vm,Rm,Tm), whereAll subtask sets representing task vehicle m, use/>Representing subtasks/>And subtask/>Dependency relationship between subtasks/>Is a subtask/>Preamble node, task/>After completion, task/>Can be performed,/>For subtask/>And subtask/>Data traffic between the two, T m, is the maximum tolerated delay of the computing task.
As shown in fig. 2, each circle represents a subtask node, edges (a, b) represent the predecessor nodes of subtasks a and b, and the values of edges (a, b) represent the data traffic between subtasks, the dependency between subtasks being irreversible. Each subtask node is represented asWherein/>Data quantity (bits) representing subtask node i of vehicle m,/>Representing the number of CPU cycles required to process 1bit data of a task, i.e., the computational resources,/>, andRepresenting the maximum tolerant delay of the subtask node,/>Representing earliest offload start time for the subtask node,/>Indicating the latest offload completion time for that subtask node.
In step S1, the method for dividing all subtasks in the computing task into critical subtasks and non-critical subtasks comprises the following specific steps, namely S11-S13.
S11, calculating the earliest starting time and the latest finishing time of each subtask, wherein the calculation formula is as follows:
In the method, in the process of the invention, Representing subtasks/>Is a subsequent sub-task of (a); /(I)Representing subtasks/>Is a preamble sub-task of (2); /(I)Representing subtasks/>Is a communication delay of (1); for subtasks/>, without a predecessor subtask，/>Is 0; for subtasks without subsequent subtasks/>，/>To calculate the maximum tolerated time delay of the task, i.e., T m.
In order to improve the unloading reliability and consider the limited service vehicle, the invention divides the subtasks into critical subtasks and non-critical subtasks and adopts different unloading strategies to unload the subtasks. Thus, step S12 is performed.
S12, calculating the urgent degree of each subtask, wherein the expression formula is as follows:
In the method, in the process of the invention, Representing subtasks/>Is the urgency of (a); /(I)For subtask/>Average unload time of/>The calculation formula of (2) is as follows:
Wherein n is the maximum number of service vehicles in the service vehicle set; f j is the free computing resources of the j-th service vehicle.
S13, comparing the urgency degree of each subtask with a preset threshold value; and when the urgency degree of the subtask is not higher than the preset threshold, defining the subtask as a critical subtask, otherwise defining the subtask as a non-critical subtask.
Step S13 may be understood as a subtask satisfying the following conditions is defined as a critical subtask:
In the invention, when the subtask is unloaded and fails, for the critical subtask, if the service vehicle is reselected for the second unloading, if The time after the sub-task re-offload is completed may be greater than the latest completion time, which may result in a computing task failure. Therefore, in order to improve the reliability of unloading, the invention unloads the key subtasks to two service vehicles in a copying way when unloading the key subtasks, thereby improving the unloading success rate and the reliability. In addition, the invention adds sigma as a margin to the above conditional formula to reduce these effects, taking into account the variability of service vehicles, the variability of service vehicles' idle resources, and the communication delay, the sigma value range being (0, 1), the present embodiment sigma taking 0.5. For non-critical subtasks, only a second time of offloading subtasks when the subtask offload fails.
S2, respectively constructing a kinematic model, a communication model, a subtask calculation delay model, a subtask execution priority model and a link reliability model between the task vehicle and the service vehicle, and accordingly establishing a constraint problem for generating an optimal unloading strategy.
In this embodiment, step S2 includes the following specific steps, S21-S26.
S21, constructing a kinematic model. The speed difference between the task vehicle and the service vehicle may cause the distance between the two workshops to increase/decrease, resulting in connection interruption beyond the maximum communication range of the task vehicle, resulting in task failure. The offloading process is only completed when the service vehicle and the mission vehicle remain within V2V communication after the up-going and execution phases are completed. Therefore, the invention needs to calculate the distance between the task vehicle and the service vehicle at the moment when the sub-task execution stage is completed, if the value is smaller than the V2V maximum communication range, the sub-task is successfully unloaded, and as each automobile has respective speed and acceleration, the distance expression formula between the task vehicle and the service vehicle is as follows:
Wherein D m,n (t) represents the distance between the task vehicle m and the service vehicle n at the time t when the sub-task execution phase is completed; d m,n(t0) represents the distance between the task vehicle m and the service vehicle n at the off-load task start time t 0, D m,n(t0)=xm(t0)－xn(t0),xm(t0) and x n(t0) represent the positions of the task vehicle m and the service vehicle n at t 0, respectively; deltav (t 0)=vm(t0)－vn(t0),vm(t0) and v n(t0) represent the speeds of mission vehicle m and service vehicle n at t 0, respectively; Δa (t 0)=am(t0)－an(t0),am(t0) and a n(t0) represent the acceleration of the mission vehicle m and the service vehicle n at t 0, respectively.
In reality, however, the behavior of each car is independent, and the task vehicle only knows the speed of the service vehicle at the time when the off-load task starts, and cannot determine the speed of the service vehicle between the starting time and the task execution completion time. Therefore, the predicted distance D m,n (t) is only a reference factor.
S22, constructing a communication model; wherein the transmission delay of the task vehicle for transmitting the subtasks to the service vehicle through the uploading channel is as followsAnd/>; Wherein R m,n represents the transmission rate of the mission vehicle m to the service vehicle n; representing subtasks/> The size of the data volume transmitted to the service vehicle n is defined by the subtask/>Data volume per se/>And the sum of the data amounts transmitted by the preamble subtasks.
Wherein, the transmission rate R m,n can be obtained by shannon formula:
Wherein B m,n represents the bandwidth of the upload channel; p m denotes the transmission power of the mission vehicle m; Represents the path loss between the service vehicle and the mission vehicle, delta represents the path loss factor, and l represents the distance between the service vehicle and the mission vehicle; h represents the channel fading factor of the uploading link; n 0 represents gaussian white noise power.
Further, since the size of the output data of the calculation task is generally much smaller than the size of the input data, the present embodiment ignores the result return time in calculating the communication delay.
S23, constructing a subtask calculation delay model; wherein the subtasksDelay in servicing vehicle offload tasksAnd/>; Subtask/>The time required to complete the unloading process is/>And (2) and; Subtask/>Unload start time of/>Subtask/>Is (1) the unloading end time isAnd/>，/>; In the/>For subtask/>The unload end time of the preamble subtask.
Wherein, when the subtaskWhen the first node of the computing task is the first node, then the subtask/>Is 0; besides, subtasks/>Is the maximum unload complete time for all of its immediate predecessor subtasks.
S24, constructing a subtask execution priority model; wherein subtasks of task vehicle m are definedFor subtask/>Preamble subtasks of/>Execution priority of (3) is higher than/>；/>Is/>And (2) and; Where pri (p) is the subtask/>Execution priority of the preamble subtask.
Wherein, when a subtask node is unloaded, the priority of the node becomes 1. When the execution priority of the subtask node becomes 0, the node can perform unloading calculation. By a first subtask of the mission vehicle mThe priority of each subtask node is recursively calculated throughout the computing task. As can be seen from the above execution priority formula, when the execution priority of all the direct-predecessor subtasks of a node becomes 1, the execution priority of the node is 0, and the unloading calculation can be performed.
S25, constructing a link reliability model. When the number of service vehicles with idle resources in the environment exceeds the current required number of sub-tasks to be offloaded, the invention uses the link reliability as an index to select the service vehicles to be offloaded. The reliability of the link can ensure the stability of the unloading process and reduce the possibility of sub-node task failure and retransmission. Wherein, in unloading subtasksWhen the link reliability P m,n,i between the service vehicle n and the mission vehicle m is expressed as follows:
Wherein e is a natural constant; t theroy is the theoretical time for the service vehicle to travel out of the communication range of the task vehicle m at the current speed; as can be seen from the above link reliability formulas, when the speed and acceleration of the mission vehicle m and the service vehicle n are identical at time t 0, the link connection is considered to be very reliable, the larger the value of P m,n,i, the better the link reliability between vehicles, i.e., the ratio of t theroy to t The larger the link reliability is, the better the link reliability is.
S26, establishing constraint problems according to the model constructed in the steps S21-S25. The invention fully utilizes the computing resources of the service vehicle by offloading the subtask nodes to the service vehicle available in the Internet of vehicles environment. Due to the complexity and diversity of the vehicle network environment, there are multiple subtask nodes to be offloaded and multiple service vehicles available, and at the same time, each subtask node is different, and the idle computing resources and driving conditions of each service vehicle are also different. Thus, to maximize the offloading success rate of the computing tasks, the present invention determines an offloading policy based on the characteristics of each service vehicle and each subtask node. This problem is a linear integer programming problem, which has been demonstrated to be NP-HARD (non-deterministic polynomial), in order to maximize the offloading success rate, the present invention describes the constraint problem as:
Wherein D success represents the number of calculation tasks successfully unloaded; constraint C1 represents the last subtask completion time of task vehicle m The maximum tolerance time delay of the whole calculation task is not exceeded; constraint C2 indicates that initial free resource F n serving vehicle n is between most free resource F max and least free resource F min; constraint C3 represents subtask/>Execution priority/>Lower than the lowest execution priority/>, among all its predecessor subtasks; Constraint C4 represents a subtaskUnloading start time/>Not earlier than the latest unload end time/>, of all its predecessor subtasks。
For the NP-hard problem, the prior art is usually solved by using a heuristic algorithm, however, most of the heuristic algorithms in the prior art are unstable in the actual vehicle network, and cannot realize fast decision when facing the large-scale problem. The deep reinforcement learning algorithm is considered as an effective method for searching the optimal strategy in the complex dynamic system, and can be solved by modeling the problem as a Markov decision process and then by an algorithm based on the deep reinforcement learning algorithm, see step S3 in detail.
And S3, modeling the constraint problem as a Markov decision process, solving the Markov decision process by adopting a deep reinforcement learning algorithm to obtain an optimal unloading strategy, and unloading subtasks to be unloaded of the task vehicle to the service vehicle according to the optimal unloading strategy.
In the internet of vehicles environment defined in this embodiment, a group of task vehicles are set to offload the computing tasks with the dependency relationships. Because the computing resources of the vehicle itself cannot support completion of the computing task within the maximum tolerable latency of the computing task, the task vehicle needs to offload the computing task to a service vehicle within communication range. The invention aims to maximize the unloading success rate of a computing task by specifying an optimal unloading strategy. At the system start time, the task vehicle gathers information about all service vehicles within its communication range, inserts one service vehicle as a member into the vehicle queue C, each member of the queue comprising four elementsAnd respectively representing the position, acceleration and speed of the idle computing resource at the current moment. In addition, each subtask that calculates the task weight can also be described by eight elementsThe input data size, the computing resources required by each bit, the maximum tolerance time delay, the earliest starting time, the latest finishing time, the task urgency degree, the execution priority and the task unloading destination are respectively represented. The tuple for each subtask is placed in the task queue Q.
The Markov decision process includes: state space, action space, and reward functions.
State space: the invention takes the information parameters of the service vehicle in the communication range and the state parameters of the self-calculation task at the current moment as the input states of a deep reinforcement learning algorithm and expresses the parameters as follows:
s(t’)=[C(t’),Q(t’)]
Wherein s (t ') is an input state at any time t'; c (t ') is t', said vehicle queue being a member of the service vehicle; and (3) a task queue when Q (t ') is t', wherein the task queue is taken as a member by the subtask.
In the vehicle queue C (t'), each service vehicle member includes four elements, denoted:
where x n(t')、an(t')、vn (t') represents the position, acceleration, and speed, respectively, of the service vehicle n at the current time.
In the task queue Q (t'), each subtask member includes eight elements, denoted:
In the method, in the process of the invention, Representing subtasks/>Maximum tolerated time delay of (2); choose denotes a service vehicle to which the subtask is offloaded.
In the unloading process of a computing task, when one subtask returns a computing result, recalculating the execution priority of the not-scheduled subtask in the computing task; when the service vehicle does not return an unloading result within the expected completion time, judging that the unloading is failed, changing the execution priority of the corresponding subtasks to 0, and putting the subtasks into an unloading queue again to wait for calculation; wherein, for a critical subtask, the maximum value of the predicted completion times offloaded to two service vehicles is taken as the predicted completion time of the critical subtask; when the offloading of the subtask is completed, the execution priority of the subtask becomes 1.
The action space is used for selecting a service vehicle for the subtasks; for a subtask with execution priority of 0, the subtask can be unloaded, so that the subtask can be unloaded to a service vehicle for calculation; all action sets of the action space are represented as action= {1,2, …, n },1,2, …, n being the number of the service vehicle.
The reward function is used for maximizing the success rate of the calculation task, and consists of the following three parts:
(1) Time delay: with latest offload completion time of subtask nodes Subtracting the actual completion time t true indicates that the greater the value the more computing resources the subtask node chooses to serve the vehicle. Furthermore, if the predecessor node of the subtask node fails to offload, this may result in an increase in the predicted completion time of the subtask node, which may result in a decrease in the latency portion.
(2) Importance level: for different subtask nodes, the importance degree is different according to the urgency degree, and the higher the urgency degree is, the higher the importance degree on the computing task is, and the higher the influence degree on the computing task delay is. The initial attribute for each subtask node is a constant value.
(3) Link reliability: in the invention, the reliability of the link is positively correlated with the reliability of task unloading, and the higher the reliability of the link is, the lower the probability of failure of the subtask node is.
Thus, the calculation formula for the bonus function is:
Wherein r i (t') represents that the task vehicle schedules the subtasks at t The awards obtained; t true is a subtaskIs calculated as/>。
In this embodiment, the deep reinforcement learning algorithm in step S3 may be DDPG algorithm (DEEP DETERMINISTIC Policy Gradient, depth deterministic Policy Gradient algorithm).
DDPG is an algorithm for outputting deterministic actions, whose structural form mainly includes policy network, target policy network, value network and target value network.
Among them, the policy Network (Actor Network) is a neural Network for learning deterministic policies in DDPG algorithm. It receives as input the state of the environment and outputs a deterministic action. The goal of the policy network is to maximize the jackpot to find the optimal policy. During training, the policy network updates the parameters by a gradient ascent method so that the expected jackpot increases.
The target policy network (Target Actor Network) is a copy of the policy network for generating the target action. Its parameters are updated periodically from the policy network by Soft Update (Soft Update) to remain more stable. The existence of the target strategy network helps to reduce estimation errors of the target value network, thereby improving training stability.
A value Network (Critic Network) is a neural Network used to estimate a cost function of a state-action pair. It receives as input the state and action of the environment, outputs a corresponding action value function, representing the expected jackpot under a given state and action. The goal of the value network is to minimize the mean square error to approximate the cumulative value of the true rewards.
The target value Network (TARGET CRITIC Network) is a copy of the value Network for generating the target value. Similar to the target policy network, the parameters of the target value network are also updated periodically from the value network by soft updating. The objective value network is used for reducing the estimation error of the action value function, so that the stability of algorithm training is improved.
It should be noted that DDPG also employs an experience replay technique to store the sampled experiences in order to the experience pool, each of which consists of information of current state s, action a taken, received reward r, next state s' and termination state, and retains a certain amount of recent experience. During the training process, a batch of experience samples are randomly extracted from the experience pool as training data. This helps break the temporal correlation between samples, preventing the negative impact of data correlation on algorithm learning. The method accelerates the convergence speed of the algorithm, improves the stability of exploration and strategy, and finally improves the performance of the algorithm.
In depth deterministic strategy gradients, the Markov Decision Process (MDP) is a mathematical framework used to describe reinforcement learning problems, including state space, action space, reward functions, transition probabilities, and the like. In applying DDPG algorithm, the MDP is used to describe the state transition and rewards mechanism of the problem, thereby guiding the agent to learn and make decisions.
Specifically, how the Markov decision process is applied in the DDPG algorithm is as follows:
1. State space (STATE SPACE): all possible states are defined in the MDP, representing the current situation of the environment. In the DDPG algorithm, the state space is typically represented by a vector or image that is used to describe the current observations of the environment.
2. Action space (Action space): all actions that an agent can take are defined in the MDP. The DDPG algorithm outputs deterministic actions through the policy network so that the intelligent agent can make decisions directly in the continuous action space.
3. Bonus function (Reward function): the rewards function in the MDP defines the immediate rewards that the agent gets during the state transition. The DDPG algorithm adjusts the policy network based on the environmental return prize signal to maximize the future jackpot.
4. Transition probability (Transition probabilities): MDP describes the probability that an agent will transition to the next state after taking some action in the state space. In the DDPG algorithm, new states and rewards are obtained by interacting with the environment without explicit modeling transition probabilities.
5. Value function (Value function): the cost function in the MDP is used to evaluate the value of the state or state-action pair, guiding the agent in making decisions. In the DDPG algorithm, a value function network (Critic) is used to estimate the value of the state-action pairs, helping to optimize the policy network.
By integrating the various components of the MDP into the DDPG algorithm, the agent can continually learn and refine the strategy based on the state, rewards, and value information of the environment to achieve a long-term cumulative maximum rewards. In this process, the theoretical framework of the Markov decision process directs the agent to make a reasonable decision in the reinforcement learning task, thereby achieving efficient learning.
The invention applies DDPG algorithm to obtain the optimal unloading strategy, guides the learning process of an Agent by setting a proper reward function, and the DDPG algorithm combines the ideas of an experience playback mechanism, strategy gradient and Q learning (Q-learning), updates the neural network parameters by maximizing the accumulated reward value and continuously optimizes the strategy by utilizing optimization algorithms such as gradient descent and the like so as to approach the optimal strategy.
Referring to fig. 3 and 4, the present embodiment considers a multi-lane one-way road, where a plurality of vehicles travel in the same direction, the vehicle travel speed range is [0,80] km/h, the subtask number range of the calculation task is [8,25], the V2V communication bandwidth is 10mhz, the V2V maximum communication range is 50-100m, and the number of service vehicles in the task vehicle communication range is 5-10.
For the model constructed in the invention, the initial input data is design data, which is the initial state of DDPG algorithm, and changes with the progress of time and execution of actions in DDPG algorithm. The input data includes two parts, a vehicle part and a calculation task part. The vehicle part comprises the following elements, namelyAnd respectively representing the idle computing resources, the position, the acceleration and the speed of the vehicle, wherein the range of the idle computing resources is 4-5GHz. The computing task section contains the following elements,/>The input data size, the computing resources required by each bit, the maximum tolerance time delay, the earliest starting time, the latest finishing time, the task urgency degree, the execution priority and the task unloading destination are respectively represented. Wherein/>Is in the range of 600-1200KB,/>In the range of 1500-2500 megaweeks,/>Is in the range of 1-5s. The four parts of earliest starting time, latest finishing time, task urgency and execution priority can be obtained by the calculation formula. The output data is in two parts: one part is the return (rewards) per round of the DDPG algorithm when training the model; the other part is whether the delay and unloading obtained after the simulation experiment are successful or not by using the trained model after the training is finished, and the part is the final required result.
Referring to fig. 5 to 10, in order to embody the superiority of the unloading scheme proposed by the present invention over the existing scheme, it is verified by comparing with the following three schemes:
(1) No duplication: the scheme does not replicate subtasks, and other contents are the same as the scheme provided by the invention.
(2) Greedy algorithm replication: the scheme uses a greedy algorithm for copy unloading, and the rest is consistent with the scheme provided by the invention.
(3) Maximum subtask replication: the scheme selects the subtasks requiring more computing resources from all the subtasks to copy, and other contents are consistent with the scheme provided by the invention.
In fig. 5, the number of subtasks of the task is set to 25, and the communication distance, i.e., the communication range, is 50 meters. As the number of service vehicles increases, subtasks have more offload options, and the available computing resources increase, resulting in increased task success rates.
In fig. 6, the service vehicles are 7 vehicles, and the communication distance is 50 meters. As the number of subtasks increases, the completion time of the dependent tasks increases and the likelihood of the service vehicle driving out of communication range increases. The increase in computational resources required to rely on tasks results in subtasks being offloaded to lower-ranked service vehicles due to the limited number of service vehicles. In short, as the number of subtasks increases, the task success rate decreases.
In fig. 7, the number of service vehicles is 7, and the number of subtasks is 25. With the increase of the communication distance, the probability that the service vehicle exits the communication range is reduced, and the success rate of depending on tasks is improved.
In fig. 8, the number of service vehicles is 7, and the communication distance is 50 meters. As the number of service vehicles increases, the mission vehicles may select a better service vehicle to offload, thereby reducing the average delay time. Meanwhile, due to the difference of task success rates, the average calculation delay of the unloading strategy adopted by the embodiment is superior to that of other strategies.
In fig. 9, the number of service vehicles is 7, and the communication distance is 50 meters. With the increase of the number of subtasks, the dependent task completion time also increases, and the average delay of the four strategies is different for the same subtask due to the different task success rates and the different computing resource usage of the four strategies.
In fig. 10, there are 7 service vehicles and 25 subtasks. As the communication distance increases, the probability of the service vehicle exiting the communication range decreases, the task success rate increases, and the average delay decreases.
In summary, compared with the three schemes, the invention has obvious advantages of task success rate and average delay under different conditions.
Example 2
The present embodiment provides a computer terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor.
The computer terminal may be a smart phone, a tablet computer, a notebook computer, etc. capable of executing a program. The processor may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to execute the program code stored in the memory or process the data. The processor, when executing the program, implements the steps of the dependent task offloading method in the V2V scenario in embodiment 1.
Example 3
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the dependent task offload method in the V2V scenario in embodiment 1.
The computer readable storage medium may include flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk provided on the computer device, a smart memory card (SMART MEDIA CARD, SMC), a secure digital (SecureDigital, SD) card, a flash memory card (FLASH CARD), or the like. Of course, the storage medium may also include both internal storage units of the computer device and external storage devices. In this embodiment, the memory is typically used to store an operating system and various application software installed on the computer device. In addition, the memory can be used to temporarily store various types of data that have been output or are to be output.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.

Claims (10)

1. The method for unloading the dependent tasks in the V2V scene is characterized by comprising the following steps of:
s1, taking the urgency degree of a plurality of subtasks with dependency relations in a task vehicle calculation task as a division standard, and dividing all the subtasks in the calculation task into a critical subtask and a non-critical subtask;
when the key subtasks are unloaded, the key subtasks are copied and unloaded to two service vehicles; when non-critical subtasks are unloaded and the unloading is failed, unloading the subtasks which are unloaded failed for the second time;
s2, respectively constructing a kinematic model, a communication model, a subtask calculation delay model, a subtask execution priority model and a link reliability model between the task vehicle and the service vehicle, and accordingly establishing a constraint problem for generating an optimal unloading strategy;
and S3, modeling the constraint problem as a Markov decision process, solving the Markov decision process by adopting a deep reinforcement learning algorithm to obtain an optimal unloading strategy, and unloading subtasks to be unloaded of the task vehicle to the service vehicle according to the optimal unloading strategy.
2. The method for offloading dependent tasks in a V2V scenario according to claim 1, wherein in step S1, the method for dividing all subtasks in the computing task into critical subtasks and non-critical subtasks comprises the following specific steps:
s11, calculating the earliest unloading starting time and the latest unloading ending time of each subtask, wherein the calculation formula is as follows:
In the/> Representing the ith subtask/>, of the mission vehicle mIs the latest unloading end time of (3); /(I)Representing subtasks/>Is the earliest unload start time in (a); /(I)Representing subtasks/>Is a subsequent sub-task of (a); /(I)Representing subtasks/>Is a preamble sub-task of (2); /(I)Representing subtasks/>Data volume of/>Representing processing subtasks/>Computing resources required for each bit of data; f max represents the most free resources of the service vehicle n; /(I)Representing subtasks/>Is a communication delay of (1); for subtasks/>, without a predecessor subtask，/>Is 0; for subtasks without subsequent subtasks/>，/>The maximum tolerance time delay of the task is calculated;
S12, calculating the urgent degree of each subtask, wherein the expression formula is as follows:
In the/> Representing subtasks/>Is the urgency of (a); /(I)For subtask/>Is a mean unloading time of (2);
S13, comparing the urgency degree of each subtask with a preset threshold value; and when the urgency degree of the subtask is not higher than the preset threshold, defining the subtask as a critical subtask, otherwise defining the subtask as a non-critical subtask.
3. The method for offloading dependent tasks in a V2V scenario as claimed in claim 2, wherein the subtasks areAverage unload time of/>The calculation formula of (2) is as follows:
wherein n is the maximum number of service vehicles in the service vehicle set; f j is the free computing resources of the j-th service vehicle.
4. The method for offloading dependent tasks in a V2V scenario according to claim 2, wherein step S2 comprises the specific steps of:
s21, constructing a kinematic model; the distance expression formula between the task vehicle and the service vehicle is as follows:
Wherein D m,n (t) represents the distance between the task vehicle m and the service vehicle n at the time t when the sub-task execution phase is completed; d m,n(t0) represents the distance between the task vehicle m and the service vehicle n at the start time t 0 of subtask offloading, D m,n(t0)=xm(t0)－xn(t0),xm(t0) and x n(t0) represent the positions of the task vehicle m and the service vehicle n at t 0, respectively; deltav (t 0)=vm(t0)－vn(t0),vm(t0) and v n(t0) represent the speeds of mission vehicle m and service vehicle n at t 0, respectively; Δa (t 0)=am(t0)－an(t0),am(t0) and a n(t0) represent the acceleration of the mission vehicle m and the service vehicle n at t 0, respectively;
S22, constructing a communication model; wherein the transmission delay of the task vehicle for transmitting the subtasks to the service vehicle through the uploading channel is as follows And/>; Wherein R m,n represents the transmission rate of the mission vehicle m to the service vehicle n; /(I)Representing subtasks/>The size of the data volume transmitted to the service vehicle n is defined by the subtask/>The sum of the data volume of the self and the data volume transmitted by the preamble subtask thereof;
S23, constructing a subtask calculation delay model; wherein the subtasks Delay in servicing vehicle offload tasks is/>And/>; Subtask/>The time required to complete the unloading process is/>And/>; Subtask/>Unload start time of/>Subtask/>Is/>And (2) and，/>; In the/>For subtask/>The unloading end time of the preamble subtask;
s24, constructing a subtask execution priority model; wherein subtasks of task vehicle m are defined For subtask/>Preamble subtasks of/>Execution priority of (3) is higher than/>；/>Is/>And (2) and; Where pri (p) is the subtask/>Execution priority of the preamble subtask;
s25, constructing a link reliability model; wherein, in unloading subtasks When the link reliability P m,n,i between the service vehicle n and the mission vehicle m is expressed as follows:
Wherein e is a natural constant; t theroy is the theoretical time for the service vehicle to travel out of the communication range of the task vehicle m at the current speed; the value range of P m,n,i is (0, 1);
S26, establishing a constraint problem as shown below according to the model constructed in the steps S21-S25:
wherein D success represents the number of calculation tasks successfully unloaded; constraint C1 represents the last subtask completion time/>, of task vehicle m The maximum tolerant time delay T m of the whole calculation task is not exceeded; constraint C2 indicates that initial free resource F n serving vehicle n is between most free resource F max and least free resource F min; constraint C3 represents subtask/>Execution priority/>Lower than the lowest execution priority among all its predecessor subtasks; constraint C4 represents subtask/>Unloading start time/>Not earlier than the latest unload end time of all its predecessor subtasks; In constraint C5,/>And/>Representing a set of task vehicles and a set of service vehicles, respectively.
5. The method for offloading dependent tasks in a V2V scenario as claimed in claim 4, wherein in step S22, the calculation formula of the transmission rate R m,n is:
Wherein B m,n represents the bandwidth of the upload channel; p m denotes the transmission power of the mission vehicle m; /(I) Represents the path loss between the service vehicle and the mission vehicle, delta represents the path loss factor, and l represents the distance between the service vehicle and the mission vehicle; h represents the channel fading factor of the uploading link; n 0 represents gaussian white noise power.
6. The method for offloading dependent tasks in a V2V scenario according to claim 4, wherein in step S3, said markov decision process comprises: state space, action space, and reward functions;
the state space is used for representing information parameters of the service vehicle in the communication range of the task vehicle and state parameters of the self-calculation task at the current moment at each moment, and the parameters are used as input states of a deep reinforcement learning algorithm and are represented as s (t ')= [ C (t '), Q (t ') ]; wherein s (t ') is an input state at any time t'; c (t ') is t', said vehicle queue being a member of the service vehicle; a task queue when Q (t ') is t', wherein the task queue is formed by subtasks as members;
In the unloading process of a computing task, when one subtask returns a computing result, recalculating the execution priority of the not-scheduled subtask in the computing task; when the service vehicle does not return an unloading result within the expected completion time, judging that the unloading is failed, changing the execution priority of the corresponding subtasks to 0, and putting the subtasks into an unloading queue again to wait for calculation; wherein, for a critical subtask, the maximum value of the predicted completion times offloaded to two service vehicles is taken as the predicted completion time of the critical subtask; when the unloading of the subtask is completed, the execution priority of the subtask is changed to 1;
the action space is used for selecting a service vehicle for the subtasks; for a subtask with execution priority of 0, the subtask can be unloaded, so that the subtask can be unloaded to a service vehicle for calculation; all action sets of the action space are represented as action= {1,2, …, n }, where 1,2, …, n are the numbers of the service vehicles;
The calculation formula of the reward function is ; Where r i (t ') represents the task vehicle's scheduling subtasks/>, at tThe awards obtained; t true is subtask/>Is calculated according to the following formula。
7. The method of dependent task offloading in a V2V scenario of claim 6, wherein in the vehicle queue C (t'), each service vehicle member comprises four elements, expressed as:
Wherein x n(t')、an(t')、vn (t') represents the position, acceleration and speed of the service vehicle n at the current time, respectively;
In the task queue Q (t'), each subtask member includes eight elements, denoted:
In the/> Representing subtasks/>Maximum tolerated time delay of (2); choose denotes a service vehicle to which the subtask is offloaded.
8. The method for task offloading of claim 7, wherein in step S3, the deep reinforcement learning algorithm is DDPG algorithm.
9. Computer terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, realizes the steps of a method of dependency task offloading in a V2V scenario according to any one of claims 1 to 8.
10. A computer readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the steps of a dependent task offload method in a V2V scenario as claimed in any one of claims 1 to 8.
CN202410316670.8A 2024-03-20 2024-03-20 Dependency task unloading method, terminal and storage medium in V2V scene Active CN117939535B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410316670.8A CN117939535B (en) 2024-03-20 2024-03-20 Dependency task unloading method, terminal and storage medium in V2V scene

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410316670.8A CN117939535B (en) 2024-03-20 2024-03-20 Dependency task unloading method, terminal and storage medium in V2V scene

Publications (2)

Publication Number Publication Date
CN117939535A true CN117939535A (en) 2024-04-26
CN117939535B CN117939535B (en) 2024-05-31

Family

ID=90763383

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410316670.8A Active CN117939535B (en) 2024-03-20 2024-03-20 Dependency task unloading method, terminal and storage medium in V2V scene

Country Status (1)

Country Link
CN (1) CN117939535B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021012584A1 (en) * 2019-07-25 2021-01-28 北京工业大学 Method for formulating single-task migration strategy in mobile edge computing scenario
CN113316116A (en) * 2021-05-27 2021-08-27 南京邮电大学 Vehicle calculation task unloading method based on multi-arm gambling machine
US20220032933A1 (en) * 2020-07-31 2022-02-03 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for generating a task offloading strategy for a vehicular edge-computing environment
US20220317692A1 (en) * 2022-06-23 2022-10-06 Intel Corporation Computational task offloading
US20220400062A1 (en) * 2021-06-15 2022-12-15 Guangdong Polytechnic Normal University METHOD AND APPARATUS FOR DIFFERENTIALLY OPTIMIZING QUALITY OF SERVICE QoS
US20230081937A1 (en) * 2020-05-22 2023-03-16 Huawei Technologies Co., Ltd. Computation offloading method and communication apparatus
CN116455903A (en) * 2023-05-10 2023-07-18 杭州电子科技大学 Method for optimizing dependency task unloading in Internet of vehicles by deep reinforcement learning
CN116541106A (en) * 2023-07-06 2023-08-04 闽南理工学院 Computing task unloading method, computing device and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021012584A1 (en) * 2019-07-25 2021-01-28 北京工业大学 Method for formulating single-task migration strategy in mobile edge computing scenario
US20230081937A1 (en) * 2020-05-22 2023-03-16 Huawei Technologies Co., Ltd. Computation offloading method and communication apparatus
US20220032933A1 (en) * 2020-07-31 2022-02-03 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for generating a task offloading strategy for a vehicular edge-computing environment
CN113316116A (en) * 2021-05-27 2021-08-27 南京邮电大学 Vehicle calculation task unloading method based on multi-arm gambling machine
US20220400062A1 (en) * 2021-06-15 2022-12-15 Guangdong Polytechnic Normal University METHOD AND APPARATUS FOR DIFFERENTIALLY OPTIMIZING QUALITY OF SERVICE QoS
US20220317692A1 (en) * 2022-06-23 2022-10-06 Intel Corporation Computational task offloading
CN116455903A (en) * 2023-05-10 2023-07-18 杭州电子科技大学 Method for optimizing dependency task unloading in Internet of vehicles by deep reinforcement learning
CN116541106A (en) * 2023-07-06 2023-08-04 闽南理工学院 Computing task unloading method, computing device and storage medium

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
BENHONG ZHANG: "Task offloading and resource allocation for intersection scenarios in vehicular edge computing", 《INTERNATIONAL JOURNAL OF SENSOR NETWORKS》, 1 June 2023 (2023-06-01) *
MUTSUMI TOYODA; HAYATA SATAKE;: "A Dynamic Task Offloading Method with Centralized Controller to Improve Task Success Rate", 《2021 IEEE INTERNATIONAL CONFERENCE ON PERVASIVE COMPUTING AND COMMUNICATIONS WORKSHOPS AND OTHER AFFILIATED EVENTS (PERCOM WORKSHOPS)》, 24 May 2021 (2021-05-24) *
XIANG BI: "A back adjustment based dependent task offloading scheduling algorithm with fairness constraints in VEC networks", 《COMPUTER NETWORKS》, 2 January 2023 (2023-01-02) *
XIANG BI: "An RSU-crossed dependent task offloading scheme for vehicular edge computing based on deep reinforcement learning", 《INTERNATIONAL JOURNAL OF SENSOR NETWORKS》, 3 May 2023 (2023-05-03) *
卢海峰;顾春华;罗飞;丁炜超;杨婷;郑帅;: "基于深度强化学习的移动边缘计算任务卸载研究", 计算机研究与发展, no. 07, 7 July 2020 (2020-07-07) *
张建军: "车联网中基于任务紧急性的联合卸载方案", 《电子测量与仪器学报》, 15 November 2020 (2020-11-15) *
童钊: "移动边缘计算中多约束下的任务卸载和资源分配算法", 《计算机工程与科学》, 15 October 2020 (2020-10-15) *
莫继为: "边缘计算网络中任务感知的服务缓存算法研究", 《硕士电子期刊》, 15 January 2023 (2023-01-15) *

Also Published As

Publication number Publication date
CN117939535B (en) 2024-05-31

Similar Documents

Publication Publication Date Title
Qi et al. Knowledge-driven service offloading decision for vehicular edge computing: A deep reinforcement learning approach
Chen et al. Efficiency and fairness oriented dynamic task offloading in internet of vehicles
CN113645273B (en) Internet of vehicles task unloading method based on service priority
Talpur et al. DRLD-SP: A deep-reinforcement-learning-based dynamic service placement in edge-enabled internet of vehicles
Qi et al. Vehicular edge computing via deep reinforcement learning
CN112672382A (en) Hybrid collaborative computing unloading method and device, electronic equipment and storage medium
Lv et al. Edge computing task offloading for environmental perception of autonomous vehicles in 6G networks
CN114173301B (en) Safe and efficient data sharing method for Internet of vehicles based on DAG block chain
CN116204315A (en) Track-based dynamic task unloading method for vehicle resource pool in Internet of vehicles
CN111539534A (en) General distributed graph processing method and system based on reinforcement learning
CN117939535B (en) Dependency task unloading method, terminal and storage medium in V2V scene
Yu et al. A situation enabled framework for energy-efficient workload offloading in 5G vehicular edge computing
CN116916272A (en) Resource allocation and task unloading method and system based on automatic driving automobile network
Zhang et al. Cooperative task offloading in cybertwin-assisted vehicular edge computing
CN115065683B (en) Vehicle edge network task allocation and unloading method based on vehicle clustering
CN116489668A (en) Edge computing task unloading method based on high-altitude communication platform assistance
Huang et al. Learning-aided fine grained offloading for real-time applications in edge-cloud computing
Wei et al. TBOMC: A task-block-based overlapping matching-coalition scheme for task offloading in vehicular fog computing
CN111866810B (en) Internet of vehicles frequency spectrum allocation method and equipment
Tao et al. Digital Twin and DRL-Driven Semantic Dissemination for 6G Autonomous Driving Service
Farimani et al. Deadline-aware task offloading in vehicular networks using deep reinforcement learning
CN113448707A (en) Online batch parallel task scheduling method in edge computing
Zhang et al. Novel resource allocation algorithm of edge computing based on deep reinforcement learning mechanism
Omland Deep Reinforcement Learning for Computation Offloading in Mobile Edge Computing
CN118250308B (en) Method, terminal and server for computing and unloading middle-element reinforcement learning edges of Internet of vehicles

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant