CN114862263A - Multi-region logistics motorcade delivery management method for improving renewable energy consumption - Google Patents
Multi-region logistics motorcade delivery management method for improving renewable energy consumption Download PDFInfo
- Publication number
- CN114862263A CN114862263A CN202210599966.6A CN202210599966A CN114862263A CN 114862263 A CN114862263 A CN 114862263A CN 202210599966 A CN202210599966 A CN 202210599966A CN 114862263 A CN114862263 A CN 114862263A
- Authority
- CN
- China
- Prior art keywords
- action
- vehicle
- charging
- renewable energy
- vehicles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012384 transportation and delivery Methods 0.000 title claims abstract description 25
- 238000007726 management method Methods 0.000 title claims abstract description 23
- 238000005265 energy consumption Methods 0.000 title claims abstract description 17
- 230000009471 action Effects 0.000 claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000013528 artificial neural network Methods 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 claims abstract description 8
- 230000002787 reinforcement Effects 0.000 claims abstract description 5
- 230000006399 behavior Effects 0.000 claims abstract description 3
- 230000006870 function Effects 0.000 claims description 9
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 claims 1
- 238000010187 selection method Methods 0.000 claims 1
- 238000007599 discharging Methods 0.000 abstract description 4
- 230000006872 improvement Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000010248 power generation Methods 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000002716 delivery method Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06312—Adjustment or analysis of established resource schedule, e.g. resource or task levelling, or dynamic rescheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Game Theory and Decision Science (AREA)
- Educational Administration (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a multi-region logistics motorcade distribution management method for improving renewable energy consumption, which specifically comprises the following steps: establishing a multi-region logistics vehicle distribution management decision into a Markov decision process; in reinforcement learning, an Actor-Critic method (A2C) is adopted to learn the delivery management strategy of a fleet; selecting an intelligent agent action by adopting two neural networks, and evaluating the state value of the intelligent agent; and an experience pool is adopted to stably and quickly evaluate the training process and the training result of the network. The invention provides a multi-region logistics motorcade delivery management method for improving renewable energy consumption, which adopts an enhanced learning A2C algorithm to learn a delivery strategy of vehicles in a multi-region; the charging behavior of the vehicle is guided through the charging service price based on the output of the renewable energy, the consumption of the renewable energy is improved, meanwhile, the efficient distribution of the logistics vehicle is guaranteed, and the charging and discharging scheduling flexibility of the electric vehicle is exerted.
Description
Technical Field
The invention relates to an electric logistics vehicle scheduling decision method considering renewable energy consumption in an electric power system, in particular to a multi-region logistics fleet delivery management method for improving renewable energy consumption.
Background
Because the operating vehicles are more sensitive to time cost, the electric operating fleet is more prone to charging at a quick charging station, and the access of a large number of electric vehicles in a short time may cause other adverse effects on the power distribution network, such as local overload, over-voltage and the like. However, the access of large-scale electric vehicles to the power grid also provides new possibilities for improving the flexibility of scheduling resources of charging and discharging facilities, and particularly increases the consumption of renewable energy resources of charging stations. In a large-scale electric logistics motorcade, a delivery system based on electric vehicles provides a delivery method for ensuring effective management of the motorcade and flexible scheduling of charging behaviors according to scheduling characteristics of the delivery system. The logistics vehicles of multizone can receive system high in the clouds dispatch, respond to charging station pricing guide, charge in renewable energy power generation period to keep the normal efficient delivery of motorcade.
Chinese patent publication No. CN109693573A discloses an electric vehicle cluster charging power optimization method for promoting renewable energy consumption, which is based on an electric vehicle collector operation model, establishes a multi-objective optimization problem, and solves the optimal charging strategy of a vehicle and the optimal configuration of EVA through a two-stage algorithm; chinese patent publication No. CN106160091A discloses a charging and discharging scheduling method for an electric vehicle battery replacement station for promoting renewable energy consumption, which maximizes the renewable energy consumption through two time scales in the day and the day, and proposes a charging and discharging plan and a day-ahead output plan of the battery replacement station; the Chinese patent publication No. CN110598904A provides a vehicle network energy interaction optimization method considering renewable energy consumption in a market environment, based on a market transaction environment and considering the renewable energy consumption, a coordination problem is built into a double-layer planning model, the upper layer problem realizes the optimal pricing of an electric vehicle charging station, the lower layer problem determines a travel route and a charging decision, and the solution is realized through strong-dual theorem conversion.
Disclosure of Invention
In order to solve the technical problems mentioned above, the invention provides a multi-region logistics fleet delivery management method for improving renewable energy consumption.
The design purpose of the invention is implemented by the following technical scheme:
the utility model provides a multizone logistics motorcade delivery management method for improving renewable energy consumption, which comprises the following steps:
s1: from the aspect of multi-region logistics vehicle scheduling, dividing a scheduling region into a plurality of hexagons; in order to cooperate with charging, order matching and distribution problems of logistics vehicles in multiple regions under the condition of considering renewable energy output, a motorcade distribution management problem is modeled into a Markov decision problem, and a delivery management strategy of a motorcade is learned by adopting an enhanced learning Actor-Critic method (A2C); setting the vehicle as an agent, wherein the state of the agent comprises a vehicle state and environment information, an action space A is {0,1,2}, an action a is 0 represents that the vehicle selects charging, and an action a is 1 represents that the vehicle selects to accept a new order; the action a is 2, the vehicle selects delivery; the reward function R includes the charging cost and the revenue of the order obtained after the vehicle action is selected at each moment.
S2: calculating an action state value Q (s, a) aiming at each action according to an Actor in an action space of the agent to obtain an initial action corresponding to the maximum action state value; for all the vehicles which are selected to be charged, assuming that a charging pile of a charging station is enough to meet the charging requirement of a fleet, for all the vehicles which select new orders, calculating the state value V(s) of the intelligent agent after order matching is performed, selecting the order with the maximum state value, for all the vehicles which are selected to be delivered, calculating the state value of the vehicles after the vehicles perform the action by utilizing Critic, and selecting the delivery action with the maximum corresponding state value; the action selection process is repeated until all vehicles have selected an action.
S3: the adopted AC architecture of the deep reinforcement learning algorithm comprises two neural networks, one is a neural Network Actor Network for selecting actions, and the other is a neural Network critical Network for evaluating state values.
As a further improvement, the continuous time is dispersed into T discrete time periods, the duration of each time step is 15 minutes, the scheduling area is divided into G areas, and the learning of the space-time law of environment information transmission by the intelligent agent is facilitated.
As a further improvement, two neural networks are respectively adopted as an intelligent action selection Network and an evaluation Network, wherein the output dimension of the Actor Network is 3, and the action state values of three actions are respectively taken under the state s.
As a further improvement, the introduction of the dominance function instead of the original reward in Critic network can be used as an index for measuring the quality of the selected action value and the average value of all actions, and the dominance function of taking action a in state s is a (s, a) ═ Q (s, a) -v(s).
The invention provides a multi-region logistics motorcade distribution management method for improving renewable energy consumption, which has the technical effects that:
the method is based on multi-category decisions of multi-region logistics vehicle distribution, a distribution decision process of a fleet is modeled into a Markov decision process, and renewable energy sources are considered to exert power in a charging station. The distribution management decision of the logistics vehicles in the multiple regions is guided based on the electricity price obtained by the renewable energy power generation, so that the renewable energy consumption level in the charging station is improved, the efficient operation of the logistics motorcade is kept, and the rebalance of the charging requirements of the motorcade in different charging regions and the real-time distribution management of the motorcade in the multiple regions are realized.
Drawings
Fig. 1 is a flowchart for establishing a multi-region logistics vehicle delivery scheduling decision as a markov decision according to the present invention.
FIG. 2 is a flowchart of a multi-zone vehicle distribution management algorithm provided by the present invention.
Fig. 3 is a flowchart of the network parameter training based on the A2C framework provided in the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings.
Referring to fig. 1, the modeling of the process for establishing the multi-zone logistics vehicle delivery scheduling decision as a markov decision provided by the present invention is as follows:
s11: converting the multi-region logistics vehicle scheduling problem into a Markov decision problem, and learning a delivery management strategy of a fleet by adopting an Actor-Critic method (A2C) based on an Advantage function through reinforcement learning;
s12: considering the output condition of the renewable energy, charging by using the renewable energy in a time period with the output of the renewable energy, setting the corresponding charging service price to be 0, and setting the vehicle to be an intelligent agent, wherein the state s of the intelligent agent comprises the vehicle state and the environment information;
s13: the action space A is {0,1,2}, the action a is 0 to represent that the vehicle selects charging, the action a is 1 to represent that the vehicle selects to accept a new order, the action a is 2 to represent that the vehicle selects matching, the reward function R comprises charging cost and order profit obtained after the vehicle action is selected at each moment, and R is an instant reward obtained after the agent finishes executing the action at each moment.
Referring to fig. 2, the flow of the multi-zone vehicle distribution management algorithm provided by the present invention is as follows:
s21: under the state s, the intelligent agent calculates the action state value Q (s, a) of each action according to the Actor for each action a in the action space, and obtains the initial action corresponding to the maximum action state value;
s22: for all vehicles selected to be charged, the charging pile of the charging station is assumed to be enough to meet the charging requirement of the fleet;
s23: for all vehicles selecting new orders, calculating a state value V (s ') of the intelligent agent after order matching is executed by utilizing Critic, wherein s' is a state obtained by transferring after the intelligent agent executes the action a in the state s, and selecting a logistics order with the maximum state value;
s24: for all the vehicles selected for delivery, calculating the state value of the vehicles after the vehicles perform the action by utilizing Critic for all the delivery actions, and selecting the delivery action with the maximum corresponding state value;
s25: the action selection process is repeated until all vehicles have selected an action.
Referring to fig. 3, the network parameter training method based on the A2C framework provided by the present invention includes the following steps:
s31: initializing parameters of an Actor network and a Critic network, namely theta and phi respectively
S32: according to the state s and the action a obtained after the intelligent agent interacts with the environment, the state value transmitted by the evaluation network is also obtained
And
updating network parameters with a function Loss (theta), where
S33: according to interaction between the intelligent agent and the environment, obtaining an interactive information state s, an action a, an obtained instant reward r and a state s' of the next moment, and storing the interactive information into an experience pool;
s34: sampling from a pool of experiences using a loss function
Updating network parameters, wherein
The multi-region logistics fleet delivery management method for improving renewable energy consumption provided by the embodiment of the invention is described in detail above, and the principle of the invention is described herein by using specific examples for illustrating the core idea of the invention, which should not be construed as limiting the scope of the invention.
Claims (4)
1. A multi-region logistics motorcade delivery management method for improving renewable energy consumption is characterized by comprising the following steps:
s1: from the aspect of multi-region logistics vehicle scheduling, dividing a scheduling region into a plurality of hexagons; in order to cooperate with charging, order matching and distribution problems of logistics vehicles in multiple regions under the condition of considering renewable energy output, a motorcade distribution management problem is modeled into a Markov decision problem, and a delivery management strategy of a motorcade is learned by adopting an enhanced learning Actor-Critic method (A2C); setting the vehicle as an agent, wherein the state of the agent comprises a vehicle state and environment information, an action space A is {0,1,2}, an action a is 0 represents that the vehicle selects charging, and an action a is 1 represents that the vehicle selects to accept a new order; the action a is 2, the vehicle selects delivery; the reward function R comprises charging cost and order income obtained after vehicle action selection at each moment;
s2: calculating an action state value Q (s, a) aiming at each action according to an Actor in an action space of the agent to obtain an initial action corresponding to the maximum action state value; for all the vehicles which are selected to be charged, assuming that a charging pile of a charging station is enough to meet the charging requirement of a fleet, for all the vehicles which select new orders, calculating the state value V(s) of the intelligent agent after order matching is performed, selecting the order with the maximum state value, for all the vehicles which are selected to be delivered, calculating the state value of the vehicles after the vehicles perform the action by utilizing Critic, and selecting the delivery action with the maximum corresponding state value; the action selection process is repeated until all the vehicles finish selecting actions;
s3: the adopted AC architecture of the deep reinforcement learning algorithm comprises two neural networks, one is a neural Network Actor Network for selecting actions, and the other is a neural Network critical Network for evaluating state values.
2. The method of claim 1, wherein the multi-zone logistics vehicle motion space modeling of step S1, comprises: the vehicle distribution management aiming at multiple regions can carry out unified modeling on multiple types of behaviors of the vehicle, and is beneficial to multiple decisions of the cooperative vehicle.
3. The method of claim 1, the action selection method of step S2, wherein: the selection from the initial action to the final concrete action utilizes both the action network to select the initial action and the evaluation network to select the final concrete action.
4. The method of claim 1, the strategy fitting method of step S2, wherein: a reinforcement learning method based on a value function and an action strategy is combined, and an experience pool is introduced to improve the training speed of the value function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210599966.6A CN114862263A (en) | 2022-05-30 | 2022-05-30 | Multi-region logistics motorcade delivery management method for improving renewable energy consumption |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210599966.6A CN114862263A (en) | 2022-05-30 | 2022-05-30 | Multi-region logistics motorcade delivery management method for improving renewable energy consumption |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114862263A true CN114862263A (en) | 2022-08-05 |
Family
ID=82641870
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210599966.6A Pending CN114862263A (en) | 2022-05-30 | 2022-05-30 | Multi-region logistics motorcade delivery management method for improving renewable energy consumption |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114862263A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118095992A (en) * | 2024-04-23 | 2024-05-28 | 厦门佳语源电子商务有限公司 | Method and device for inserting bills for same-day distribution in crowdsourcing mode |
-
2022
- 2022-05-30 CN CN202210599966.6A patent/CN114862263A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118095992A (en) * | 2024-04-23 | 2024-05-28 | 厦门佳语源电子商务有限公司 | Method and device for inserting bills for same-day distribution in crowdsourcing mode |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109508857B (en) | Multi-stage planning method for active power distribution network | |
| CN110649641B (en) | Electric automobile quick charging station energy storage system and method based on source network charge storage cooperative service | |
| Li et al. | Emission-concerned wind-EV coordination on the transmission grid side with network constraints: Concept and case study | |
| CN112186809B (en) | Virtual power plant optimization cooperative scheduling method based on V2G mode of electric vehicle | |
| CN109559035A (en) | A kind of urban power distribution network bi-level programming method considering flexibility | |
| CN107104454A (en) | Meter and the optimal load flow node electricity price computational methods in electric automobile power adjustable control domain | |
| CN106058855A (en) | Active power distribution network multi-target optimization scheduling method of coordinating stored energy and flexible load | |
| CN112217195B (en) | Cloud energy storage charging and discharging strategy forming method based on GRU multi-step prediction technology | |
| CN102708425A (en) | Coordination control system and method for electric vehicle service network based on Multi-Agent system | |
| CN113869678B (en) | Capacity planning method for electric vehicle charging system in shared energy storage mode | |
| CN112800658A (en) | Active power distribution network scheduling method considering source storage load interaction | |
| CN112670999B (en) | Low-voltage distribution network real-time voltage control method based on user-side flexible resources | |
| CN113326467A (en) | Multi-station fusion comprehensive energy system multi-target optimization method based on multiple uncertainties, storage medium and optimization system | |
| CN114914923A (en) | Grid method based variable-time-length two-stage electric vehicle scheduling method and system | |
| CN114862263A (en) | Multi-region logistics motorcade delivery management method for improving renewable energy consumption | |
| CN105205561A (en) | Expressway charging station real-time interaction electricity price setting method based on mobile energy carrying | |
| CN114462854A (en) | Hierarchical scheduling method and system containing new energy and electric vehicle grid connection | |
| Lin et al. | Aggregator pricing and electric vehicles charging strategy based on a two-layer deep learning model | |
| Sridharan et al. | A hybrid approach based energy management for building resilience against power outage by shared parking station for EVs | |
| CN117791625A (en) | Ordered charge and discharge planning method, equipment and medium for electric automobile | |
| CN112721706B (en) | Capacity optimization method of electric vehicle charging station energy storage system considering elasticity | |
| CN117172861A (en) | Mobile charging dynamic pricing method based on user load difference and space constraint | |
| CN112396223A (en) | Electric vehicle charging station energy management method under interactive energy mechanism | |
| CN116596252A (en) | Multi-target charging scheduling method for electric automobile clusters | |
| CN116054286A (en) | Residential area capacity optimal configuration method considering multiple elastic resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |