CN114362221A - Regional smart power grid partition evaluation method based on deep reinforcement learning - Google Patents
Regional smart power grid partition evaluation method based on deep reinforcement learning Download PDFInfo
- Publication number
- CN114362221A CN114362221A CN202210048549.2A CN202210048549A CN114362221A CN 114362221 A CN114362221 A CN 114362221A CN 202210048549 A CN202210048549 A CN 202210048549A CN 114362221 A CN114362221 A CN 114362221A
- Authority
- CN
- China
- Prior art keywords
- partition
- evaluation index
- time
- power
- microgrid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005192 partition Methods 0.000 title claims abstract description 101
- 238000011156 evaluation Methods 0.000 title claims abstract description 98
- 230000002787 reinforcement Effects 0.000 title claims abstract description 22
- 230000008859 change Effects 0.000 claims abstract description 14
- 238000000638 solvent extraction Methods 0.000 claims abstract description 13
- 230000000694 effects Effects 0.000 claims abstract description 10
- 238000000034 method Methods 0.000 claims abstract description 8
- 230000007246 mechanism Effects 0.000 claims abstract description 4
- 230000009471 action Effects 0.000 claims description 24
- 238000004146 energy storage Methods 0.000 claims description 22
- 230000003993 interaction Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 238000010248 power generation Methods 0.000 claims description 9
- 230000003247 decreasing effect Effects 0.000 claims description 6
- 230000008878 coupling Effects 0.000 claims description 4
- 238000010168 coupling process Methods 0.000 claims description 4
- 238000005859 coupling reaction Methods 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 230000005611 electricity Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- VYMDGNCVAMGZFE-UHFFFAOYSA-N phenylbutazonum Chemical compound O=C1C(CCCC)C(=O)N(C=2C=CC=CC=2)N1C1=CC=CC=C1 VYMDGNCVAMGZFE-UHFFFAOYSA-N 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 239000003795 chemical substances by application Substances 0.000 description 18
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 4
- 229910052799 carbon Inorganic materials 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 2
- 238000011217 control strategy Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000003345 natural gas Substances 0.000 description 1
- 238000006386 neutralization reaction Methods 0.000 description 1
- 239000003209 petroleum derivative Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013316 zoning Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/28—Arrangements for balancing of the load in a network by storage of energy
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0206—Price or cost determination based on market factors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/12—Circuit arrangements for ac mains or ac distribution networks for adjusting voltage in ac networks by changing a characteristic of the network load
- H02J3/14—Circuit arrangements for ac mains or ac distribution networks for adjusting voltage in ac networks by changing a characteristic of the network load by switching loads on to, or off from, network, e.g. progressively balanced loading
- H02J3/144—Demand-response operation of the power transmission or distribution network
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
- H02J3/381—Dispersed generators
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
- H02J3/46—Controlling of the sharing of output between the generators, converters, or transformers
- H02J3/466—Scheduling the operation of the generators, e.g. connecting or disconnecting generators to meet a given demand
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
- H02J2203/20—Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
- H02J2300/20—The dispersed energy generation being of renewable origin
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
- H02J2300/40—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation wherein a plurality of decentralised, dispersed or local energy generation technologies are operated simultaneously
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B70/00—Technologies for an efficient end-user side electric power management and consumption
- Y02B70/30—Systems integrating technologies related to power network operation and communication or information technologies for improving the carbon footprint of the management of residential or tertiary loads, i.e. smart grids as climate change mitigation technology in the buildings sector, including also the last stages of power distribution and the control, monitoring or operating management systems at local level
- Y02B70/3225—Demand response systems, e.g. load shedding, peak shaving
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S20/00—Management or operation of end-user stationary applications or the last stages of power distribution; Controlling, monitoring or operating thereof
- Y04S20/20—End-user application control systems
- Y04S20/222—Demand response systems, e.g. load shedding, peak shaving
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S50/00—Market activities related to the operation of systems integrating technologies related to power network operation or related to communication or information technologies
- Y04S50/14—Marketing, i.e. market research and analysis, surveying, promotions, advertising, buyer profiling, customer management or rewards
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Health & Medical Sciences (AREA)
- Power Engineering (AREA)
- Accounting & Taxation (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Finance (AREA)
- Evolutionary Computation (AREA)
- Tourism & Hospitality (AREA)
- Mathematical Physics (AREA)
- Educational Administration (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Game Theory and Decision Science (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Supply And Distribution Of Alternating Current (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a regional intelligent power grid partition evaluation method based on deep reinforcement learning, which comprises the following steps of: (1) establishing a multi-microgrid system partition evaluation index system; (2) setting power balance limit of a multi-microgrid system; (3) establishing an evaluation index weight intelligent agent; (4) constructing an evaluation index system of the intra-area and interval division effect; (5) designing a multi-microgrid partition comprehensive evaluation index; (6) and designing a multi-microgrid re-partitioning mechanism considering system node change. According to the regional intelligent power grid regional evaluation method, multiple indexes are considered, and a regional intelligent power grid regional evaluation index system is more comprehensive than the existing regional intelligent power grid regional indexes; according to the invention, a deep reinforcement learning method is adopted, and each index weight is determined based on historical data of each node, so that the influence of monitoring bad data on the weight is more easily resisted, and the robustness is stronger; the invention can update and adjust the partition evaluation index system in time according to the state change of the network node so as to ensure the reasonability and the harmony of the partition evaluation index.
Description
Technical Field
The invention relates to the field of cross technology application of electric power system theory and artificial intelligence, in particular to a regional intelligent power grid partition evaluation method based on deep reinforcement learning.
Background
At present, traditional energy sources such as petroleum and natural gas are gradually exhausted, and the problems of environmental pollution, overproof carbon emission and the like in development and utilization of the traditional energy sources are increasingly severe. The development of renewable energy sources and the construction of a micro-grid are effective ways for promoting green transformation of an energy system and achieving the goals of carbon peak reaching and carbon neutralization. In a regional intelligent power grid system which is composed of a plurality of micro-grids and contains high-proportion renewable energy, a distributed power supply is connected with a load and an energy storage device through an energy router, and energy is enabled to be changed from unidirectional supply to bidirectional interaction. Due to the development of the internet of things sensing technology and the power electronic technology, various distributed renewable energy sources can be effectively integrated with a regional intelligent power grid system and can quickly respond along with the change of load requirements. But the access of the controllable equipment greatly increases the decision complexity of the energy management of the regional intelligent power grid system and the dimension of the system control problem. Therefore, under the scene containing high-proportion renewable energy, in order to reduce the overall operation and maintenance cost of the smart grid system to the greatest extent, how to design a reasonable energy scheduling and management and control strategy promotes the renewable energy to be fully consumed on the spot, improves the energy utilization efficiency and reduces the carbon emission becomes a problem to be solved urgently. And dimension disasters can be encountered by directly implementing an energy scheduling and control strategy aiming at the regional smart grid scene of high-proportion renewable energy sources. Therefore, the dimension of the control problem needs to be reduced, and the control problem needs to be decomposed into a plurality of sub-problems, that is, the overall control problem of the regional smart grid system is converted into the control problems of a plurality of sub-regions, and the regional smart grid system is partitioned, so that the local area autonomous is in the first place, and the wide area interconnection is in the second place. But how do reasonably efficient partitioning? The determination of zone boundaries of a regional smart grid system is typically dependent on certain zone metrics. The construction of a reasonable, comprehensive and efficient partition index system is particularly important.
The traditional regional smart grid partitioning index has the following defects and shortcomings:
(1) the partitioning index is too simple. Only considering system power distribution, energy flow and geographic factors, lacking comprehensive consideration of various indexes, and being incapable of being effectively matched with actual conditions. Multiple indexes need to be brought into an evaluation system so as to carry out scientific and reasonable evaluation on the partition method.
(2) Conventional zoning indicators generally take into account factors such as line loss and power balance. However, with the development of energy storage technology in recent years, more and more energy storage devices are used in regional smart grid systems. Conventional partition indicators cannot effectively cope with such a situation.
(3) The weight value is set manually, and an error exists. The traditional partitioning index is usually based on an index weighted value set manually, and has an error with the actual condition, so that the subsequent partitioning result is influenced.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide a regional intelligent power grid partition evaluation method based on deep reinforcement learning, which has intelligence, coordination, sociality and self-learning.
The technical scheme is as follows: the invention relates to a regional intelligent power grid partition evaluation method based on deep reinforcement learning, which comprises the following steps of:
(1) establishing a multi-microgrid system partition evaluation index system, wherein the multi-microgrid system partition evaluation index system comprises an intra-area evaluation index system and an interval evaluation index system;
(1.1) establishing an intra-area evaluation index system of the multi-microgrid system partition, wherein evaluation indexes in the intra-area evaluation index system comprise:
the utilization rate of renewable energy sources:Pc(t) actual generated power of renewable energy source at time t, Pst(t) is the rated power generation power of the renewable energy source at the moment t, and t is the moment of the multi-microgrid system;
supply and demand balance degree in the area:Pprc(t) the power generated at the energy supply side at time t, Pu(t) is the user load at time t;
adjustable capacity of distributed power supply: ein,3=Pst(t)-Pc(t),Pc(t) actual generated power of renewable energy source at time t, Pst(t) rated power generation of renewable energy at time t;
degree of electrical coupling between nodes: ein,4When energy interaction exists among the nodes, the value is 1; when no interaction exists between the nodes, the value is 0;
matching degree of load and distributed power supply:Pc(t) actual generated power of renewable energy source at time t, Pu(t) is the user load at time t;
the energy storage device can adjust capacity: ein,6=Pba,st(t)-Pba,c(t),Pba,c(t) actual generated power of the energy storage device at time t, Pba,stAnd (t) is the rated power generation power of the energy storage device at the moment t.
(1.2) establishing an interval evaluation index system of the multi-microgrid system partition, wherein evaluation indexes in the interval evaluation index system comprise:
average electrical distance between sections:Lcir,iis the electrical distance, Σ L, of this partition from the i-th surrounding partitioncir,iIs the sum of the electrical distances of the local and surrounding partitions, ncirThe number of surrounding partitions;
interval power interaction capacity: eex,2=∑Pcir,i(t),Pcir,i(t) the power of the interaction between the local partition and the i-th surrounding partition at time t, sigma Pcir,i(t) is the sum of the interactive power of the local partition and the surrounding partitions at t time;
power scheduling cost: eex,3The electric energy loss generated when the local subarea and the surrounding subareas interact with each other by 1kWh electric energy is obtained;
line loss: eex,4The sum of the electric energy loss generated by the power interaction of the partition and the surrounding partitions;
whether each sub-area is connected with a backbone network: eex,5When the partition is connected with the backbone network, the value of the partition is 1; when the partition is not connected with the backbone network, the value of the partition is 0;
energy storage equipment breaking cost: eex,6Energy storage device losses per 1kWh of electrical energy stored/released by the energy storage device;
Ein,iand Eex,iAre the main components of the corresponding intra-area and extra-area evaluation indexes.
(2) Setting power balance limit of a multi-microgrid system; the power balance relation corresponding to the power balance limit in the step (2) is as follows:
Ptrans(t)=Pprc(t)-Pu(t)
wherein, Ptrans(t) denotes the interaction power between the microgrid and the backbone network at time t, Pprc(t) represents the power generated at the energy supply side at time t, PuAnd (t) represents the user coincidence power at the time t.
(3) Establishing an evaluation index weight intelligent agent;
(3.1) setting the observation state quantity required by the intelligent agent: the method comprises the indexes in the steps (1) and (2), and the state space is as follows:
st∈S:{Ein,i,t,Eex,i,t}
in the state space S, Ein,i,tRepresenting the state quantity of the indicator in the ith zone at time t, Eex,i,tIndicates the state quantity, s, of the i-th section index at time ttThe state of the intelligent agent at the time t is represented, and the time t represents the time of the multi-microgrid system;
(3.2) setting the action value of the agent: the actions of the agent include increasing the weight, decreasing the weight and not changing, and the action space is as follows:
at∈A{0,1,2}
in the motion space a, 0 represents decreasing weight, 1 represents increasing weight, and 2 represents no change;
(3.3) setting the reward function of the agent when the agent is in state stTaking action oftThe obtained reward is the operation cost of the multi-microgrid system at the time t, and is as follows:
rt=Ptrans×Rt×Δt
wherein r istRepresenting the running cost R of the multi-microgrid system at the moment ttThe electricity price of interaction between the multi-microgrid system and the backbone network at the moment t is represented, and delta t represents an action time interval;
(3.4) simulating the action of the agent by using a deep neural network, inputting the state quantity in the action space into the deep neural network, and outputting an index weight value lambda (s, a) in the observation state by the deep neural network:
λ(s,a)=E(rt+λ(st+1,at+1)|st,at)
where λ (s, a) denotes the agent in the observed state stWhen and to take action expectations, E () represents the expectation value.
(4) Constructing an evaluation index system of the intra-area and interval division effect; the evaluation index systems of the intra-area and interval division effects in the step (4) are respectively as follows:
Gin=∑iλin,iEin,i,Gex=∑iλex,iEex,i
wherein G isinAnd GexEvaluation indexes of intra-zone and interval division effects, lambda, respectivelyin,iAnd λex,iThe higher index value represents a better dynamic division strategy for the contribution coefficient of the principal component; and the partition evaluation index system changes along with the change of the state information of the microgrid.
(5) Designing a multi-microgrid partition comprehensive evaluation index; the formula of the multi-microgrid partition comprehensive evaluation index in the step (5) is as follows:
wherein G isin,jIs the j-th intra-area evaluation index, Gex,jIs the jth interval evaluation index, and n is the number of partitions.
(6) Designing a multi-microgrid re-partitioning mechanism considering system node change; the step (6) is specifically as follows: considering the change of the nodes of the multi-microgrid system along with the environment and time, if a certain partition in the system is comprehensively evaluated, the index MIELess than 90% of the original, the multi-microgrid needs to be partitioned again; if the comprehensive evaluation index M of all the subareas in the systemIEIf the current partition strategy is better than the original 90%, the current partition strategy does not need to be re-partitioned.
A computer storage medium, on which a computer program is stored, which, when executed by a processor, implements a regional smart grid partitioning evaluation method based on deep reinforcement learning as described above.
A computer device comprises a storage, a processor and a computer program which is stored on the storage and can be operated on the reprocessor, wherein the processor executes the computer program to realize the regional intelligent power grid partition evaluation method based on deep reinforcement learning.
Has the advantages that: compared with the prior art, the invention has the following advantages:
1. the regional intelligent power grid regional evaluation index system takes multiple indexes into consideration, such as the utilization rate of renewable energy sources, the balance degree of supply and demand in a region, the adjustable capacity of a distributed power supply, the electrical coupling degree between nodes, the matching degree of a load and the distributed power supply, the adjustable capacity of energy storage equipment, the average electrical distance of the region, the interactive capacity of the region power, the power scheduling cost, the line loss, whether each sub-region is connected with a backbone network or not, the breaking cost of the energy storage equipment and the like, and the regional intelligent power grid regional evaluation index system is more comprehensive than the regional intelligent power grid regional indexes;
2. according to the invention, a deep reinforcement learning method is adopted, and the weights of all indexes are determined based on historical data of all nodes, so that the influence of monitoring bad data on the weights is more easily resisted, and the robustness is stronger;
3. the invention can update and adjust the partition evaluation index system in time according to the state change of the network node so as to ensure the reasonability and the harmony of the partition evaluation index.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
fig. 2 is a schematic diagram of partitioning a multi-microgrid system according to a partitioning evaluation index system.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
As shown in fig. 1 and 2, a regional smart grid partition evaluation method based on deep reinforcement learning includes the following steps:
(1) establishing a multi-microgrid system partition evaluation index system, wherein the multi-microgrid system partition evaluation index system comprises an intra-area evaluation index system and an interval evaluation index system;
(1.1) establishing an intra-area evaluation index system of the multi-microgrid system partition, wherein evaluation indexes in the intra-area evaluation index system comprise:
the utilization rate of renewable energy sources:Pc(t) actual generated power of renewable energy source at time t, Pst(t) isRated power generation power of the renewable energy source at the time t, wherein t is the time of the multi-microgrid system;
supply and demand balance degree in the area:Pprc(t) the power generated at the energy supply side at time t, Pu(t) is the user load at time t;
adjustable capacity of distributed power supply: ein,3=Pst(t)-Pc(t),Pc(t) actual generated power of renewable energy source at time t, Pst(t) rated power generation of renewable energy at time t;
degree of electrical coupling between nodes: ein,4When energy interaction exists among the nodes, the value is 1; when no interaction exists between the nodes, the value is 0;
matching degree of load and distributed power supply:Pc(t) actual generated power of renewable energy source at time t, Pu(t) is the user load at time t;
the energy storage device can adjust capacity: ein,6=Pba,st(t)-Pba,c(t),Pba,c(t) actual generated power of the energy storage device at time t, Pba,stAnd (t) is the rated power generation power of the energy storage device at the moment t.
(1.2) establishing an interval evaluation index system of the multi-microgrid system partition, wherein evaluation indexes in the interval evaluation index system comprise:
average electrical distance between sections:Lcir,iis the electrical distance, Σ L, of this partition from the i-th surrounding partitioncir,iIs the sum of the electrical distances of the local and surrounding partitions, ncirThe number of surrounding partitions;
interval power interaction capacity: eex,2=∑Pcir,i(t),Pcir,i(t) t time division and weekInteractive power, SIG P, around the ith partitioncir,i(t) is the sum of the interactive power of the local partition and the surrounding partitions at t time;
power scheduling cost: eex,3The electric energy loss generated when the local subarea and the surrounding subareas interact with each other by 1kWh electric energy is obtained;
line loss: eex,4The sum of the electric energy loss generated by the power interaction of the partition and the surrounding partitions;
whether each sub-area is connected with a backbone network: eex,5When the partition is connected with the backbone network, the value of the partition is 1; when the partition is not connected with the backbone network, the value of the partition is 0;
energy storage equipment breaking cost: eex,6Energy storage device losses per 1kWh of electrical energy stored/released by the energy storage device;
Ein,iand Eex,iAre the main components of the corresponding intra-area and extra-area evaluation indexes.
(2) Setting power balance limit of a multi-microgrid system; the power balance relation corresponding to the power balance limit in the step (2) is as follows:
Ptrans(t)=Pprc(t)-Pu(t)
wherein, Ptrans(t) denotes the interaction power between the microgrid and the backbone network at time t, Pprc(t) represents the power generated at the energy supply side at time t, PuAnd (t) represents the user coincidence power at the time t.
(3) Establishing an evaluation index weight intelligent agent;
(3.1) setting the observation state quantity required by the intelligent agent: the method comprises the indexes in the steps (1) and (2), and the state space is as follows:
st∈S:{Ein,i,t,Eex,i,t}
in the state space S, Ein,i,tRepresenting the state quantity of the indicator in the ith zone at time t, Eex,i,tIndicates the state quantity, s, of the i-th section index at time ttThe state of the intelligent agent at the time t is represented, and the time t represents the time of the multi-microgrid system;
(3.2) setting the action value of the agent: the actions of the agent include increasing the weight, decreasing the weight and not changing, and the action space is as follows:
at∈A{0,1,2}
in the motion space a, 0 represents decreasing weight, 1 represents increasing weight, and 2 represents no change;
(3.3) setting the reward function of the agent when the agent is in state stTaking action oftThe obtained reward is the operation cost of the multi-microgrid system at the time t, and is as follows:
rt=Ptrans×Rt×Δt
wherein r istRepresenting the running cost R of the multi-microgrid system at the moment ttThe electricity price of interaction between the multi-microgrid system and the backbone network at the moment t is represented, and delta t represents an action time interval;
(3.4) simulating the action of the agent by using a deep neural network, inputting the state quantity in the action space into the deep neural network, and outputting an index weight value lambda (s, a) in the observation state by the deep neural network:
λ(s,a)=E(rt+λ(st+1,at+1)|st,at)
where λ (s, a) denotes the agent in the observed state stWhen and to take action expectations, E () represents the expectation value.
(4) Constructing an evaluation index system of the intra-area and interval division effect; the evaluation index systems of the intra-area and interval division effects in the step (4) are respectively as follows:
Gin=∑iλin,iEin,i,Gex=∑iλex,iEex,i
wherein G isinAnd GexEvaluation indexes of intra-zone and interval division effects, lambda, respectivelyin,iAnd λex,iThe higher index value represents a better dynamic division strategy for the contribution coefficient of the principal component; and the partition evaluation index system changes along with the change of the state information of the microgrid.
(5) Designing a multi-microgrid partition comprehensive evaluation index; the formula of the multi-microgrid partition comprehensive evaluation index in the step (5) is as follows:
wherein G isin,jIs the j-th intra-area evaluation index, Gex,jIs the jth interval evaluation index, and n is the number of partitions.
(6) Designing a multi-microgrid re-partition mechanism considering system node change, wherein the step (6) is specifically as follows: considering the change of the nodes of the multi-microgrid system along with the environment and time, if a certain partition in the system is comprehensively evaluated, the index MIELess than 90% of the original, the multi-microgrid needs to be partitioned again; if the comprehensive evaluation index M of all the subareas in the systemIEIf the current partition strategy is better than the original 90%, the current partition strategy does not need to be re-partitioned.
A computer storage medium, on which a computer program is stored, which, when executed by a processor, implements a regional smart grid partitioning evaluation method based on deep reinforcement learning as described above.
A computer device comprises a storage, a processor and a computer program which is stored on the storage and can be operated on the reprocessor, wherein the processor executes the computer program to realize the regional intelligent power grid partition evaluation method based on deep reinforcement learning.
Claims (9)
1. A regional smart grid partition evaluation method based on deep reinforcement learning is characterized by comprising the following steps:
(1) establishing a multi-microgrid system partition evaluation index system, wherein the multi-microgrid system partition evaluation index system comprises an intra-area evaluation index system and an interval evaluation index system;
(2) setting power balance limit of a multi-microgrid system;
(3) establishing an evaluation index weight intelligent agent;
(4) constructing an evaluation index system of the intra-area and interval division effect;
(5) designing a multi-microgrid partition comprehensive evaluation index;
(6) and designing a multi-microgrid repartitioning mechanism.
2. The regional smart grid partition evaluation method based on deep reinforcement learning according to claim 1, wherein the step (1) specifically comprises:
(1.1) establishing an intra-area evaluation index system of the multi-microgrid system partition, wherein evaluation indexes in the intra-area evaluation index system comprise:
the utilization rate of renewable energy sources:Pc(t) actual generated power of renewable energy source at time t, Pst(t) is the rated power generation power of the renewable energy source at the moment t, and t is the moment of the multi-microgrid system;
supply and demand balance degree in the area:Pprc(t) the power generated at the energy supply side at time t, Pu(t) is the user load at time t;
adjustable capacity of distributed power supply: ein,3=Pst(t)-Pc(t),Pc(t) actual generated power of renewable energy source at time t, Pst(t) rated power generation of renewable energy at time t;
degree of electrical coupling between nodes: ein,4When energy interaction exists among the nodes, the value is 1; when no interaction exists between the nodes, the value is 0;
matching degree of load and distributed power supply:Pc(t) actual generated power of renewable energy source at time t, Pu(t) is the user load at time t;
the energy storage device can adjust capacity: ein,6=Pba,st(t)-Pba,c(t),Pba,c(t) storing energy for time tActual power generated by the plant, Pba,stAnd (t) is the rated power generation power of the energy storage device at the moment t.
(1.2) establishing an interval evaluation index system of the multi-microgrid system partition, wherein evaluation indexes in the interval evaluation index system comprise:
average electrical distance between sections:Lcir,iis the electrical distance, Σ L, of this partition from the i-th surrounding partitioncir,iIs the sum of the electrical distances of the local and surrounding partitions, ncirThe number of surrounding partitions;
interval power interaction capacity: eex,2=∑Pcir,i(t),Pcir,i(t) the power of the interaction between the local partition and the i-th surrounding partition at time t, sigma Pcir,i(t) is the sum of the interactive power of the local partition and the surrounding partitions at t time;
power scheduling cost: eex,3The electric energy loss generated when the local subarea and the surrounding subareas interact with each other by 1kWh electric energy is obtained;
line loss: eex,4The sum of the electric energy loss generated by the power interaction of the partition and the surrounding partitions;
whether each sub-area is connected with a backbone network: eex,5When the partition is connected with the backbone network, the value of the partition is 1; when the partition is not connected with the backbone network, the value of the partition is 0;
energy storage equipment breaking cost: eex,6Energy storage device losses per 1kWh of electrical energy stored/released by the energy storage device;
Ein,iand Eex,iAre the main components of the corresponding intra-area and extra-area evaluation indexes.
3. The deep reinforcement learning-based regional smart grid partition evaluation method according to claim 1, wherein the power balance relation corresponding to the power balance limit in the step (2) is as follows:
Ptrans(t)=Pprc(t)-Pu(t)
wherein, Ptrans(t) denotes the interaction power between the microgrid and the backbone network at time t, Pprc(t) represents the power generated at the energy supply side at time t, PuAnd (t) represents the user coincidence power at the time t.
4. The regional smart grid partition evaluation method based on deep reinforcement learning according to claim 1, wherein the step (3) specifically comprises:
(3.1) setting the observation state quantity required by the intelligent agent: the method comprises the indexes in the steps (1) and (2), and the state space is as follows:
st∈S:{Ein,i,t,Eex,i,t}
in the state space S, Ein,i,tRepresenting the state quantity of the indicator in the ith zone at time t, Eex,i,tIndicates the state quantity, s, of the i-th section index at time ttThe state of the intelligent agent at the time t is represented, and the time t represents the time of the multi-microgrid system;
(3.2) setting the action value of the agent: the actions of the agent include increasing the weight, decreasing the weight and not changing, and the action space is as follows:
at∈A{0,1,2}
in the motion space a, 0 represents decreasing weight, 1 represents increasing weight, and 2 represents no change;
(3.3) setting the reward function of the agent when the agent is in state stTaking action oftThe obtained reward is the operation cost of the multi-microgrid system at the time t, and is as follows:
rt=Ptrans×Rt×Δt
wherein r istRepresenting the running cost R of the multi-microgrid system at the moment ttThe electricity price of interaction between the multi-microgrid system and the backbone network at the moment t is represented, and delta t represents an action time interval;
(3.4) simulating the action of the agent by using a deep neural network, inputting the state quantity in the action space into the deep neural network, and outputting an index weight value lambda (s, a) in the observation state by the deep neural network:
λ(s,a)=E(rt+λ(st+1,at+1)|st,at)
where λ (s, a) denotes the agent in the observed state stWhen and to take action expectations, E () represents the expectation value.
5. The regional smart grid partitioning evaluation method based on deep reinforcement learning according to claim 1, wherein the evaluation index systems of the intra-regional and inter-regional partitioning effects in the step (4) are respectively as follows:
Gin=∑iλin,iEin,i,Gex=∑iλex,iEex,i
wherein G isinAnd GexEvaluation indexes of intra-zone and interval division effects, lambda, respectivelyin,iAnd λex,iThe higher index value represents a better dynamic division strategy for the contribution coefficient of the principal component; and the partition evaluation index system changes along with the change of the state information of the microgrid.
6. The regional smart grid partition evaluation method based on deep reinforcement learning of claim 1, wherein the formula of the multi-microgrid partition comprehensive evaluation index in the step (5) is as follows:
wherein G isin,jIs the j-th intra-area evaluation index, Gex,jIs the jth interval evaluation index, and n is the number of partitions.
7. The regional smart grid partition evaluation method based on deep reinforcement learning according to claim 1, wherein the step (6) specifically comprises: if a certain block of partition in the system is comprehensively evaluated to obtain an index MIELess than 90% of the original, the multi-microgrid needs to be partitioned again; if the comprehensive evaluation of all the subareas in the systemIndex MIEIf the current partition strategy is better than the original 90%, the current partition strategy does not need to be re-partitioned.
8. A computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a regional smart grid partition evaluation method based on deep reinforcement learning according to any one of claims 1 to 7.
9. A computer device comprising a storage, a processor and a computer program stored on the storage and executable on the processor, wherein the processor implements the method for regional smart grid partition evaluation based on deep reinforcement learning according to any one of claims 1 to 7 when executing the computer program.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210048549.2A CN114362221B (en) | 2022-01-17 | 2022-01-17 | Regional intelligent power grid partition evaluation method based on deep reinforcement learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210048549.2A CN114362221B (en) | 2022-01-17 | 2022-01-17 | Regional intelligent power grid partition evaluation method based on deep reinforcement learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114362221A true CN114362221A (en) | 2022-04-15 |
CN114362221B CN114362221B (en) | 2023-10-13 |
Family
ID=81092263
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210048549.2A Active CN114362221B (en) | 2022-01-17 | 2022-01-17 | Regional intelligent power grid partition evaluation method based on deep reinforcement learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114362221B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106655253A (en) * | 2016-10-26 | 2017-05-10 | 华南理工大学 | Single-/three-phase multi-micro-grid region dynamic partitioning method |
CN109255514A (en) * | 2018-07-19 | 2019-01-22 | 国网上海市电力公司 | A kind of independently-powered capability assessment method of intelligent distribution network subregion |
WO2020000399A1 (en) * | 2018-06-29 | 2020-01-02 | 东莞理工学院 | Multi-agent deep reinforcement learning proxy method based on intelligent grid |
CN111146815A (en) * | 2018-11-05 | 2020-05-12 | 西交利物浦大学 | Intelligent power distribution network distributed power generation planning configuration method |
CN113780622A (en) * | 2021-08-04 | 2021-12-10 | 华南理工大学 | Multi-micro-grid power distribution system distributed scheduling method based on multi-agent reinforcement learning |
CN113852080A (en) * | 2021-08-25 | 2021-12-28 | 中国电力科学研究院有限公司 | Method, system, equipment and storage medium for optimizing power flow of smart power grid |
-
2022
- 2022-01-17 CN CN202210048549.2A patent/CN114362221B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106655253A (en) * | 2016-10-26 | 2017-05-10 | 华南理工大学 | Single-/three-phase multi-micro-grid region dynamic partitioning method |
WO2020000399A1 (en) * | 2018-06-29 | 2020-01-02 | 东莞理工学院 | Multi-agent deep reinforcement learning proxy method based on intelligent grid |
CN109255514A (en) * | 2018-07-19 | 2019-01-22 | 国网上海市电力公司 | A kind of independently-powered capability assessment method of intelligent distribution network subregion |
CN111146815A (en) * | 2018-11-05 | 2020-05-12 | 西交利物浦大学 | Intelligent power distribution network distributed power generation planning configuration method |
CN113780622A (en) * | 2021-08-04 | 2021-12-10 | 华南理工大学 | Multi-micro-grid power distribution system distributed scheduling method based on multi-agent reinforcement learning |
CN113852080A (en) * | 2021-08-25 | 2021-12-28 | 中国电力科学研究院有限公司 | Method, system, equipment and storage medium for optimizing power flow of smart power grid |
Non-Patent Citations (1)
Title |
---|
HAOCHEN HUA等: "Towards Intelligent Energy Control and Optimization in Energy Internet: A Review", 《2019 IEEE INTERNATIONAL CONFERENCE ON ENERGY INTERNET》, pages 573 - 578 * |
Also Published As
Publication number | Publication date |
---|---|
CN114362221B (en) | 2023-10-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Yang et al. | Survey on applications of particle swarm optimization in electric power systems | |
Luh et al. | Load adaptive pricing: An emerging tool for electric utilities | |
CN105071389B (en) | The alternating current-direct current mixing micro-capacitance sensor optimizing operation method and device of meter and source net load interaction | |
CN107944757A (en) | Electric power interacted system regenerative resource digestion capability analysis and assessment method | |
CN110137955A (en) | A kind of decision-making technique counted and the robust Unit Combination of CVaR is dispatched | |
CN109038660A (en) | A kind of wind-electricity integration System Reactive Power planing method considering quiet Enhancement of Transient Voltage Stability | |
CN106529737A (en) | Planning and distribution method for peak load regulation power source on supply side of power distribution network | |
CN110362845B (en) | Method and device for determining regional power market development planning mode | |
Shendryk et al. | Decision making for energy management in smart grid | |
Bai et al. | Robust online operation of power systems with advanced adiabatic compressed air energy storage and renewable generation | |
Zheng et al. | Meta-heuristic techniques in microgrid management: A survey | |
Peng et al. | Sequential coalition formation for wind-thermal combined bidding | |
Dong et al. | Intelligent decision-making of distribution network planning scheme with distributed wind power generations | |
Mu et al. | A two-stage scheduling method for integrated community energy system based on a hybrid mechanism and data-driven model | |
Dou et al. | Double‐deck optimal schedule of micro‐grid based on demand‐side response | |
CN112510690B (en) | Optimal scheduling method and system considering wind-fire-storage combination and demand response reward and punishment | |
Yang et al. | The short-term economical operation problem for hydropower station using chaotic normal cloud model based discrete shuffled frog leaping algorithm | |
CN114362221A (en) | Regional smart power grid partition evaluation method based on deep reinforcement learning | |
CN115619159A (en) | Intelligent park energy consumption analysis scheduling method and system | |
Shah et al. | Optimization solutions for demand side management and monitoring | |
Fu et al. | Study on wind power consumption capacity of power grid considering risk and unit commitment | |
Phommixay et al. | Comparative analysis of continuous and hybrid binary-continuous particle swarm optimization for optimal economic operation of a microgrid | |
Kumari et al. | Dynamic distribution network expansion planning under energy storage integration using PSO with controlled particle movement | |
Hu et al. | Robust optimal scheduling of integrated energy systems considering multiple uncertainties | |
CN115271453B (en) | Urban raw water supply allocation path identification method, system and storable medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |