CN111654027B - 一种基于强化学习的配电物联网智能决策方法 - Google Patents

一种基于强化学习的配电物联网智能决策方法 Download PDF

Info

Publication number
CN111654027B
CN111654027B CN202010540833.2A CN202010540833A CN111654027B CN 111654027 B CN111654027 B CN 111654027B CN 202010540833 A CN202010540833 A CN 202010540833A CN 111654027 B CN111654027 B CN 111654027B
Authority
CN
China
Prior art keywords
power distribution
decision
network
reinforcement learning
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010540833.2A
Other languages
English (en)
Other versions
CN111654027A (zh
Inventor
蒋超
单华
宋庆武
徐妍
蒋峰
李春鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Fangtian Power Technology Co Ltd
Original Assignee
Jiangsu Fangtian Power Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Fangtian Power Technology Co Ltd filed Critical Jiangsu Fangtian Power Technology Co Ltd
Priority to CN202010540833.2A priority Critical patent/CN111654027B/zh
Publication of CN111654027A publication Critical patent/CN111654027A/zh
Application granted granted Critical
Publication of CN111654027B publication Critical patent/CN111654027B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
    • H02J13/00001—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by the display of information or by user interaction, e.g. supervisory control and data acquisition systems [SCADA] or graphical user interfaces [GUI]
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
    • H02J13/00002—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by monitoring
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
    • H02J13/00006—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by information or instructions transport means between the monitoring, controlling or managing units and monitored, controlled or operated power network element or electrical equipment
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J13/00—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network
    • H02J13/00006—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by information or instructions transport means between the monitoring, controlling or managing units and monitored, controlled or operated power network element or electrical equipment
    • H02J13/00028—Circuit arrangements for providing remote indication of network conditions, e.g. an instantaneous record of the open or closed condition of each circuitbreaker in the network; Circuit arrangements for providing remote control of switching means in a power distribution network, e.g. switching in and out of current consumers by using a pulse code signal carried by the network characterised by information or instructions transport means between the monitoring, controlling or managing units and monitored, controlled or operated power network element or electrical equipment involving the use of Internet protocols
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
    • H02J2203/20—Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E60/00—Enabling technologies; Technologies with a potential or indirect contribution to GHG emissions mitigation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S40/00—Systems for electrical power generation, transmission, distribution or end-user application management characterised by the use of communication or information technologies, or communication or information technology specific aspects supporting them
    • Y04S40/12—Systems for electrical power generation, transmission, distribution or end-user application management characterised by the use of communication or information technologies, or communication or information technology specific aspects supporting them characterised by data transport means between the monitoring, controlling or managing units and monitored, controlled or operated electrical equipment
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S40/00—Systems for electrical power generation, transmission, distribution or end-user application management characterised by the use of communication or information technologies, or communication or information technology specific aspects supporting them
    • Y04S40/12—Systems for electrical power generation, transmission, distribution or end-user application management characterised by the use of communication or information technologies, or communication or information technology specific aspects supporting them characterised by data transport means between the monitoring, controlling or managing units and monitored, controlled or operated electrical equipment
    • Y04S40/128—Systems for electrical power generation, transmission, distribution or end-user application management characterised by the use of communication or information technologies, or communication or information technology specific aspects supporting them characterised by data transport means between the monitoring, controlling or managing units and monitored, controlled or operated electrical equipment involving the use of Internet protocol

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Power Engineering (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Marketing (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Quality & Reliability (AREA)
  • Development Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Operations Research (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Public Health (AREA)
  • Water Supply & Treatment (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Human Computer Interaction (AREA)
  • Supply And Distribution Of Alternating Current (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本发明公开了一种基于强化学习的配电物联网智能决策方法，包括决策模型的构建方法和决策模型的应用方法，通过获取配电物联网的N个状态参数和状态参数对应的决策结果，构建样本集，将样本集输入到强化学习模型中，对强化学习模型进行训练，获取决策模型，引入数据流触发机制，采集配电物联网当前的状态参数，构成状态参数集，将状态参数集输入到决策模型中，获取当前的决策结果，将当前的决策结果发送至配电物联网的执行设备中，由执行设备对当前的决策结果进行执行；本发明提供的方法，解决了配电物联网自主感知与智能决策之间的耦合矛盾，用于配电物联网的智能决策。

Description

一种基于强化学习的配电物联网智能决策方法
技术领域
本发明涉及电气工程技术领域，具体涉及一种基于强化学习的配电物联网智能决策方法 。
背景技术
物联网是在互联网基础上延伸和扩展的网络，通过将各种信息传感设备和互联网结合起来而形成的一个巨大网络，实现在任何时间、任何地点，对人机的互联互通。配电物联网体系架构涵盖配电运行自主感知等功能，实现电网运维全寿命周期下的人机交互、时标信息提取等，为高度市场化下的电网运维提供借鉴。
配电网现有的监测体系缺乏配电设备泛在互联、配电策略全局最优自主决策方法，不能很好的解决配电物联网自主感知和智能决策之间的耦合矛盾，为电网的自动化运维带来了不便。
发明内容
本发明的目的：在于提供一种基于强化学习的配电物联网智能决策方法，解决配电物联网自主感知和智能决策之间的耦合问题。
技术方案：本发明提供的方案包括决策模型的构建方法和决策模型的应用方法，方法具体包括如下步骤：
决策模型的构建方法具体包括如下步骤：
步骤1、获取配电网的N个状态参数和所述状态参数对应的决策结果，构建样本集；所述决策结果包括是否对配电网中软硬件进行深度解耦；
步骤2、选取样本集中的M个状态参数和对应的决策结果作为训练集，选取样本集中剩余的（N-M）个状态参数和对应的决策结果作为测试集，将所述训练集和测试集输入强化学习模型中，对强化学习模型进行训练，获取决策模型；
决策模型的应用方法包括实时执行如下步骤：
步骤A、引入数据流触发机制，采集配电网当前的状态参数，构成状态参数集；
步骤B、将所述状态参数集输入决策模型中，获取当前的决策结果；
步骤C、将当前的决策结果发送至配电物联网的执行设备中；由执行设备对当前的决策结果进行执行。
还包括步骤D如下，执行完步骤C之后，进入步骤D：
步骤D、侦测执行设备是否收到干预决策并执行，如果是，将干预决策和对应的当前的状态参数的构成样本，并将构成的样本加入样本集中，对样本集进行更新；否则，样本集中的样本保持不变。
进一步的，还包括决策模型优化方法，按照预设的周期对决策模型进行优化，对决策模型的优化具体包括如下步骤：采集更新的样本集中最新更新的N个状态参数和对应的决策结果构建新的样本集，使用所述新的样本集对强化学习模型进行训练，获取优化的决策模型。
进一步的，方法还包括在配电网中安装有多个用于采集配电网状态参数的传感器；
在步骤1中，数据流触发机制为：控制***检测到配电网上电运维时，控制配电网中的各传感器对配电网进行状态参数的采集；所述状态参数包括配电网的电流参数、电压参数。
在步骤A中，方法还包括对当前采集的配电网的状态参数进行清洗。
在步骤C中，将决策结果发送至配电物联网的执行设备的方法为：采用直接序列扩频技术构建数据包多线程传输机制，融入基于Mesh协议的节点网络自组织和自愈功能，通过协调网络拓扑结构的自组网协调器实现多线程传输。
通过协调网络拓扑结构的自组网协调器实现多线程传输具体包括如下步骤：
步骤C1、多节点无线物联网上电启动后对自组网协调器的协调器软件进行初始化，并实时监测传感器集群硬件自组网是否成功；
步骤C2、开启节点网络监测机制，若监测到网络，则选择协调器或者路由节点作为初始父节点申请加入网络，否则把节点属性设置为协调器组建网络机制；
步骤C3、开启子节点入退网监测机制，如果申请入网，则根据组网需求增加子节点并为子节点分配NWK（Network Layer，网络层）参数，如果申请退网，释放原有的NWK参数关联；
步骤C4、开启多维数据传输机制，选择被测数据传输子节点，接收终端节点的多维数据进行数据上传。
优选的，强化学习模型为双重Q网络模型。
有益效果：相对于现有技术，本发明提供的方法，实现了配电设备泛在互联，解决了配电物联网自主感知和智能决策之间的耦合矛盾，有助于电网的自动化运维。
附图说明
图1是根据本发明提供实施例提供的基于双重Q网络模型的配电物联网智能决策方法逻辑图；
图2是根据本发明实施例提供的数据多线程传输流程示意图。
具体实施方式
下面结合附图对本发明作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。
参照图1，本发明提供的方法包括决策模型的构建过程：
步骤1、获取配电网的N个状态参数，以及和各状态参数对应的决策结果，构建样本集。
步骤2、随机的从步骤1中构建的样本集中选取的M个状态参数，以及这M个状态参数对应的决策结果作为训练集，将样本集中剩余的（N-M）个状态参数和对应的决策结果作为测试集，将训练集和测试集输入到强化学习模型中，对强化学习模型进行训练，训练好的强化学习模型为决策模型；
其中决策结果包括是否对配电网中的软硬件进行深度解耦、物理资源虚拟化管控方法；强化学习模型为参照图1所述的双重Q网络模型；
在本发明的实施例中，双重Q网络模型包括策略网络和估值网络，
在构建好决策模型后，对构建好的决策模型的应用方法包括实时执行如下步骤：
步骤A、在配电网中安装多个传感器，用于采集配电网的状态参数；引入了数据流触发机制，数据流触发机制为：在控制***检测到配电网上电运维时，控制配电网中的各个传感器对配电网进行状态参数的采集；传感器的包括温度传感器、湿度传感器、电流传感器、电压传感器，传感器采集的参数包括配电网的温度参数、湿度参数、电流参数、电压参数；
通过引入上述数据流触发机制，采集配电网当前的状态参数，并对采集的配电网当前的状态参数进行清洗，构成状态参数集。
步骤B、 将步骤A中构建的状态参数集输入到步骤2中构建的决策模型中，获取当前的决策结果；在进行强化学习的过程中，根据单层Q网络模型中内置的卷积核学习运行当前的状态参数，根据双层Q网络模型中内置的卷积核学习动作信息数据，实现物理资源虚拟化管控，进而获取是否对配电网软硬件进行深度解耦的决策结果。
步骤C、把步骤B中获取的决策结果发送到配电物联网的执行设备中，由执行设备对当前的决策结果进行执行；
在将获取的决策结果发送到配电物联网的执行设备中时，采用序列扩频技术构建数据包多线程传输机制，融入基于Mesh协议的节点网络自组织和自愈功能，通过协调网络拓扑结构的自组网协调器实现多线程传输。
参照图2，通过协调网络拓扑结构的自组网协调器实现多线程传输具体包括如下步骤：
步骤C1、多节点无线物联网上电启动后对自组网协调器的协调器软件进行初始化，并实时监测传感器集群硬件自组网是否成功；多节点无线物联自组网工作在470MHz或2.4GHz频段，划分为16个信道，步长值为5MHz，编号11～26，协调器通过调用信道轮询函数对信道能量波动进行实时扫描，能量水平高标志该信道无线信号活跃，协调器根据能量扫描信息选择一个可以利用的信道建立自己的无线网络实现多维数据有效传送。
步骤C2、开启节点网络监测机制，若监测到网络，则选择协调器或者路由节点作为初始父节点申请加入网络，否则把节点属性设置为协调器组建网络机制；
步骤C3、开启子节点入退网监测机制，如果申请入网，则根据组网需求增加子节点并为子节点分配NWK参数，如果申请退网，释放原有的NWK参数关联；
步骤C4、开启多维数据传输机制，选择被测数据传输子节点，接收终端节点的多维数据进行数据上传；
通过上述传输过程，可以准确有效的把智能决策结果传输到执行设备中。
步骤D、侦测执行设备是否收到干预决策并执行：若果是，将当前的干预决策和对应的当前的状态参数构成样本，并将构成的样本加入步骤1中的样本集中，对样本集进行更新；否则，不对步骤1中的样本集进行更新。
方法还包括对步骤2中所述的决策模型按照预设的周期进行优化，具体方法包括如下步骤：
从步骤D中获取的更新的样本集中，选出最新更新的N个状态参数，以及该N个状态参数对应的决策结果构建新的样本集，参照步骤1和步骤2所述的方法，对强化学习模型进行训练，获取优化的决策模型，然后跳转步骤A。
以上详细描述了本发明的具体实施例。应当理解，本领域的普通技术人员无需创造性劳动就可以根据本发明的构思作出诸多修改和变化。因此，凡本技术领域中技术人员依本发明的构思在现有技术的基础上通过逻辑分析、推理或者有限的实验可以得到的技术方案，皆应在由权利要求书所确定的保护范围内。

Claims (6)

1.一种基于强化学习的配电物联网智能决策方法，其特征在于，所述方法包括如下步骤：
决策模型的构建方法具体包括如下步骤：
步骤1、获取配电网的N个状态参数和所述状态参数对应的决策结果，构建样本集；所述决策结果包括是否对配电网中软硬件进行深度解耦；
步骤2、选取样本集中的M个状态参数和对应的决策结果作为训练集，选取样本集中剩余的(N-M)个状态参数和对应的决策结果作为测试集，将所述训练集和测试集输入强化学习模型中，对强化学习模型进行训练，获取决策模型；
决策模型的应用方法包括实时执行如下步骤：
步骤A、引入数据流触发机制，采集配电网当前的状态参数，构成状态参数集；
步骤B、将所述状态参数集输入决策模型中，获取当前的决策结果；
步骤C、将当前的决策结果发送至配电物联网的执行设备中；由执行设备对当前的决策结果进行执行；
在步骤C中，将决策结果发送至配电网的执行设备的方法为：采用直接序列扩频技术构建数据包多线程传输机制，融入基于Mesh协议的节点网络自组织和自愈功能，通过协调网络拓扑结构的自组网协调器实现多线程传输；
通过协调网络拓扑结构的自组网协调器实现多线程传输具体包括如下步骤：
步骤C1、多节点无线物联网上电启动后对自组网协调器的协调器软件进行初始化，并实时监测传感器集群硬件自组网是否成功；
步骤C2、开启节点网络监测机制，若监测到网络，则选择协调器或者路由节点作为初始父节点申请加入网络，否则把节点属性设置为协调器组建网络机制；
步骤C3、开启子节点入退网监测机制，如果申请入网，则根据组网需求增加子节点并为子节点分配NWK参数，如果申请退网，释放原有的NWK参数关联；
步骤C4、开启多维数据传输机制，选择被测数据传输子节点，接收终端节点的多维数据进行数据上传。
2.根据权利要求1所述的基于强化学习的配电物联网智能决策方法，其特征在于，还包括步骤D如下，执行完步骤C之后，进入步骤D：
步骤D、侦测执行设备是否收到干预决策并执行，若是，将干预决策和对应的当前的状态参数的构成样本，并将构成的样本加入样本集中对样本集进行更新；否则，样本集中的样本保持不变。
3.根据权利要求2所述的基于强化学习的配电物联网智能决策方法，其特征在于，还包括决策模型优化方法，按照预设的周期对决策模型进行优化，对决策模型的优化具体包括如下步骤：
采集更新的样本集中最新更新的N个状态参数和对应的决策结果构建新的样本集，使用所述新的样本集对强化学习模型进行训练，获取优化的决策模型。
4.根据权利要求1所述的基于强化学习的配电物联网智能决策方法，其特征在于，配电网中安装有多个用于采集配电网状态参数的传感器；在步骤1中，所述数据流触发机制为：控制***检测到配电网上电运维时，控制配电网中的各传感器对配电网进行状态参数的采集；所述状态参数包括配电网的电流参数、电压参数。
5.根据权利要求1所述的基于强化学习的配电物联网智能决策方法，其特征在于，在步骤3中，所述方法还包括对当前采集的配电网的状态参数进行清洗。
6.根据权利要求1至5任一项所述的基于强化学习的配电物联网智能决策方法，其特征在于，所述强化学习模型为双重Q网络模型。
CN202010540833.2A 2020-06-15 2020-06-15 一种基于强化学习的配电物联网智能决策方法 Active CN111654027B (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010540833.2A CN111654027B (zh) 2020-06-15 2020-06-15 一种基于强化学习的配电物联网智能决策方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010540833.2A CN111654027B (zh) 2020-06-15 2020-06-15 一种基于强化学习的配电物联网智能决策方法

Publications (2)

Publication Number Publication Date
CN111654027A CN111654027A (zh) 2020-09-11
CN111654027B true CN111654027B (zh) 2022-04-29

Family

ID=72349584

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010540833.2A Active CN111654027B (zh) 2020-06-15 2020-06-15 一种基于强化学习的配电物联网智能决策方法

Country Status (1)

Country Link
CN (1) CN111654027B (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112186811B (zh) * 2020-09-16 2022-03-25 北京交通大学 一种基于深度强化学习的agc机组动态优化方法
CN112670982B (zh) * 2020-12-14 2022-11-08 广西电网有限责任公司电力科学研究院 一种基于奖励机制的微电网有功调度控制方法及***

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108769942A (zh) * 2018-05-31 2018-11-06 国网江苏省电力有限公司连云港供电分公司 基于zigbee广播与组播的无线同步采样***及方法
CN111062651A (zh) * 2020-03-18 2020-04-24 南京中电科能技术有限公司 基于边缘计算和大数据分析的安全用电管理***及方法
CN111107675A (zh) * 2020-01-21 2020-05-05 山东科华电力技术有限公司 基于泛在电力物联网的电缆通道边缘物联终端及方法
CN112637883A (zh) * 2020-12-09 2021-04-09 深圳智芯微电子科技有限公司 电力物联网中对无线环境变化具有鲁棒性的联邦学习方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201628558U (zh) * 2008-12-23 2010-11-10 胡广生 一种输电线路的数字化在线监测***和监测装置
US9473583B2 (en) * 2012-12-10 2016-10-18 Linkedin Corporation Methods and systems for providing decision-making support

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108769942A (zh) * 2018-05-31 2018-11-06 国网江苏省电力有限公司连云港供电分公司 基于zigbee广播与组播的无线同步采样***及方法
CN111107675A (zh) * 2020-01-21 2020-05-05 山东科华电力技术有限公司 基于泛在电力物联网的电缆通道边缘物联终端及方法
CN111062651A (zh) * 2020-03-18 2020-04-24 南京中电科能技术有限公司 基于边缘计算和大数据分析的安全用电管理***及方法
CN112637883A (zh) * 2020-12-09 2021-04-09 深圳智芯微电子科技有限公司 电力物联网中对无线环境变化具有鲁棒性的联邦学习方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
刘中明等.基于改进深度强化学习的电力智慧供应链高纬度决策模型研究.《电子测量技术》.2019,第42卷(第23期),第53-60页. *
基于改进深度强化学习的电力智慧供应链高纬度决策模型研究;刘中明等;《电子测量技术》;20191231;第42卷(第23期);第53-60页 *
深度学习在泛在电力物联网中的应用与挑战;谢小瑜等;《电力自动化设备》;20200430;第40卷(第4期);第77-87页 *

Also Published As

Publication number Publication date
CN111654027A (zh) 2020-09-11

Similar Documents

Publication Publication Date Title
CN105684524B (zh) 通信***、接入网络节点和优化通信网络中能耗的方法和装置
Yang et al. Machine learning techniques and a case study for intelligent wireless networks
CN111654027B (zh) 一种基于强化学习的配电物联网智能决策方法
KR101421569B1 (ko) 무선 개인 통신 네트워크에서 비컨 테이블을 이용한 비컨슬롯 결정 방법 및 무선 근거리 개인 통신 기기
CN107005331B (zh) 干扰检测
Ortiz et al. Fuzzy-logic based routing for dense wireless sensor networks
CN110620611B (zh) 一种基于geo与leo双层卫星网络的协同频谱感知方法
WO2012072445A1 (en) Method and apparatus of communications
Colistra et al. Task allocation in group of nodes in the IoT: A consensus approach
Jurado-Lasso et al. A survey on machine learning software-defined wireless sensor networks (ml-SDWSNS): Current status and major challenges
WO2015062689A1 (en) A communications system and a method of determining an optimal duty cycle to minimise overall energy consumption
CN108924786B (zh) 面向环境突发事件的无线传感器网络数据采集方法
Al-Kaseem et al. A new intelligent approach for optimizing 6lowpan mac layer parameters
CN110191432B (zh) 基于物联网的智能监测***
EP3008940B1 (en) Method of coordinating a communication network
Xu et al. Data-cognition-empowered intelligent wireless networks: Data, utilities, cognition brain, and architecture
Sinclair et al. A kernel methods approach to reducing handover occurrences within LTE
CN113922899A (zh) 基于神经网络和Welch变换的频谱感知接入方法
CN107612799A (zh) 一种基于云技术应用的智能家居
CN107969008B (zh) 一种软件定义传感网集中式路由计算方法
KR20240071374A (ko) 네트워크 기반 ai(artificial intelligence) 모델 구성
CN116250270A (zh) 一种模型配置方法及装置
CN111446985B (zh) 一种面向工业无线网络的预测性抗干扰方法及网关设备
Srilakshmi et al. Selection of machine learning techniques for network lifetime parameters and synchronization issues in wireless networks
CN105392176B (zh) 一种执行器节点执行能力的计算方法

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant