CN110781224A - Data monitoring method based on electric power big data - Google Patents
Data monitoring method based on electric power big data Download PDFInfo
- Publication number
- CN110781224A CN110781224A CN201911005332.8A CN201911005332A CN110781224A CN 110781224 A CN110781224 A CN 110781224A CN 201911005332 A CN201911005332 A CN 201911005332A CN 110781224 A CN110781224 A CN 110781224A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- phase
- power
- change
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000004364 calculation method Methods 0.000 claims abstract description 27
- 230000001419 dependent effect Effects 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 abstract description 5
- 230000010354 integration Effects 0.000 abstract description 4
- 238000005065 mining Methods 0.000 abstract description 4
- 238000013480 data collection Methods 0.000 abstract 2
- 230000005611 electricity Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000013024 troubleshooting Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Tourism & Hospitality (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Administration (AREA)
- Health & Medical Sciences (AREA)
- Water Supply & Treatment (AREA)
- Probability & Statistics with Applications (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Game Theory and Decision Science (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Public Health (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Remote Monitoring And Control Of Power-Distribution Networks (AREA)
Abstract
The invention discloses a data monitoring method based on electric power big data, which belongs to the technical field of electric power and comprises the following steps: the method comprises the following steps: monitoring collected data, collecting total data of a monophase table and a related transformer area, importing the total data into hdfs, and monitoring the data collection by using spark streaming + redis; step two: monitoring data, calculating in real time, and calculating by relying on the daily electric quantity of dispatching through a dispatching program; recalculating, namely, depending on a scheduling daily electric quantity recalculating program through a scheduling program, and calculating daily electric quantities of all the days in a window. The invention improves the data quality by adopting the clustering algorithm to carry out continuous analysis, ensures the accuracy, timeliness, effectiveness and integrity of the data, provides powerful guarantee for the integration and mining application of the data, monitors the full name of the data collection, calculation and uploading, and ensures the accuracy and timeliness of the data.
Description
Technical Field
The invention relates to a data monitoring method, in particular to a data monitoring method based on electric power big data, and belongs to the technical field of electric power.
Background
Along with the development of the depth of the improvement work of the service quality of the power distribution network, higher requirements are put forward on the distribution quality and the power utilization management level of the transformer area. The three-phase line in the station is divided into lines according to the phase, but most of the single-phase lines do not store relevant phase information during filing. If the phase information of the existing line of the electric energy meter of the residential user is accurate, great convenience is brought to troubleshooting of faults and line loss abnormity and three-phase imbalance management in the transformer area. However, when the electric energy meters are installed in a large number of distribution areas through wiring, the phase information of the installed electric energy meters of the residential users is not stored, so that the installation phase information of the electric energy meters of the residential users in the distribution areas is difficult to obtain during troubleshooting, split-phase line loss treatment and three-phase imbalance treatment.
Disclosure of Invention
The invention mainly aims to provide a data monitoring method based on power big data in order to solve the defects of the prior art.
The purpose of the invention can be achieved by adopting the following technical scheme:
the data monitoring method based on the electric power big data comprises the following steps:
the method comprises the following steps: monitoring of acquired data
(1) Acquiring data, acquiring a monophase table and total data of related transformer areas, and importing the collected data into hdfs;
(2) monitoring data acquisition by spark streaming + redis;
step two: data monitoring
(1) Calculating in real time, namely calculating the daily electric quantity dependent on the dispatching through a dispatching program;
(2) recalculating, namely, relying on a scheduling daily electric quantity recalculating program through a scheduling program to calculate daily electric quantities of all the days in a window;
(3) storing the calculation results of real-time calculation and recalculation in a database and updating the calculation results into a formal library;
(4) monitoring the calculation and storage process by using spark + hbase;
step three: data upload monitoring
(1) And feeding back the data condition to the provincial and municipal metering system in real time.
Preferably, in step one, the monitoring of the collected data monitors the data at a frequency of every 15 minutes.
Preferably, in the second step, a clustering algorithm is used to continuously analyze the correlation between the single-phase table in the database and the power data change of the total table of the relevant distribution room, and the phase hooked by the single-phase table is deduced and identified.
Preferably, in the second step, the scheduler relies on the scheduling daily electricity recalculation program, and meanwhile, the scheduler transmits the parameter time, calculates the daily electricity of all the days in the window and stores the calculation end in the database according to the scheduling statistical time, and the storage process updates the latest data of the statistical time to the formal library according to the type.
Preferably, in step three, the data uploading monitoring main monitored data are files, index city level daily electric quantity and index provincial level daily electric quantity.
Preferably, in step three, the data problem is corrected in time according to the feedback of the provincial and urban metering system.
Preferably, the clustering algorithm calculation process is as follows: the power change of the single-phase table at a certain moment and the power change of a certain phase of the station area summary table at the same moment are recorded, and then the correlation between the value of the single-phase table change and the data of the station area summary table change is compared.
Preferably, the correlation between the value of the single-phase meter change and the data of the station area summary table change is comprehensively analyzed for a plurality of times, and if the low or high correlation reaches a threshold value T and the correlation of the data reaches a threshold value P, the phase hung on the single-phase meter is deduced.
Preferably, the scheduler incoming parameter time interval is T-3 to T-1.
The invention has the beneficial technical effects that: according to the data monitoring method based on the electric power big data, the relevance of the instantaneous power data fluctuation of the electric energy meter of the residential user and the relevant phase power fluctuation of the total meter of the distribution area is analyzed by acquiring the instantaneous power data of the electric energy meter in the distribution area and the edge computing capability in the metering terminal or the server computing capability in the metering automation system, thereby distinguishing the phase position hung on the single-phase electric energy meter installed by the resident user, improving the efficiency and the convenience of identifying the phase position hung on the single-phase meter, greatly saving the cost of phase position identification, meanwhile, the clustering algorithm is adopted for continuous analysis, so that the data quality is improved, the accuracy, timeliness, effectiveness and integrity of the data are guaranteed, powerful guarantee is provided for the integration and mining application of the data, the data acquisition, calculation and uploading are monitored, and the accuracy and timeliness of the data are guaranteed.
Detailed Description
In order to make the technical solutions of the present invention more clear and definite for those skilled in the art, the present invention is further described in detail with reference to the following examples, but the embodiments of the present invention are not limited thereto.
The data monitoring method based on the electric power big data provided by the embodiment comprises the following steps:
the method comprises the following steps: monitoring of acquired data
(1) Acquiring data, acquiring a monophase table and total data of related transformer areas, and importing the collected data into hdfs;
(2) monitoring data acquisition by spark streaming + redis;
step two: data monitoring
(1) Calculating in real time, namely calculating the daily electric quantity dependent on the dispatching through a dispatching program;
(2) recalculating, namely, relying on a scheduling daily electric quantity recalculating program through a scheduling program to calculate daily electric quantities of all the days in a window;
(3) storing the calculation results of real-time calculation and recalculation in a database and updating the calculation results into a formal library;
(4) monitoring the calculation and storage process by using spark + hbase;
step three: data upload monitoring
(1) And feeding back the data condition to the provincial and municipal metering system in real time.
In the first step, monitoring data is monitored at the frequency of every 15 minutes, in the second step, the relativity of the power data changes of the single-phase meter in the database and the total meter of the related station area is continuously analyzed by adopting a clustering algorithm, the phase connected with the single-phase meter is deduced and identified, in the second step, a scheduling program depends on a scheduling daily electric quantity recalculating program, meanwhile, the scheduling program transmits parameter time, all daily electric quantities in a window are calculated and stored in the database according to scheduling statistical time after the calculation is finished, the storage process updates the latest data of the statistical time to a formal base according to types, in the third step, the data are uploaded and monitored, the data are files, index market-level daily electric quantity and index provincial daily electric quantity, in the third step, the data problem is corrected in time according to the feedback of a provincial city metering system, and instantaneous power data of the electric energy meter in the station area are collected, and then through the edge computing power in the metering terminal or the server computing power in the metering automation system, the relevance of the instantaneous power data fluctuation of the electric energy meter of the residential user and the relevant phase power fluctuation of the table area master table is analyzed, so that the phases hooked by the single-phase electric energy meters installed by the residential users are distinguished, the phase efficiency and convenience of hooking the single-phase meters are improved, the phase identification cost can be greatly saved, meanwhile, the data quality is improved by adopting a clustering algorithm for continuous analysis, the accuracy, timeliness, effectiveness and integrity of the data are guaranteed, powerful guarantee is provided for the integration and mining application of the data, the full names of data acquisition, calculation and uploading are monitored, and the accuracy and timeliness of the data are guaranteed.
The clustering algorithm comprises the following calculation processes: recording the power change of a single-phase table at a certain moment and the power change of a certain phase of a station area summary table at the same moment, then comparing the correlation between the numerical value of the single-phase table change and the data of the station area summary table change, comprehensively analyzing the correlation between the numerical value of the single-phase table change and the data of the station area summary table change for many times, if the low-down correlation or the high-up correlation reaches a threshold value T and the correlation of the data reaches a threshold value P, deducing the phase hung on the single-phase table, and transmitting a scheduling program into a parameter time interval from T-3 to T-1.
The first embodiment is as follows:
the working process of the data monitoring method based on the big power data is as follows:
the method comprises the following steps: monitoring of acquired data
(1) Data acquisition, acquiring the total data of the monophase table and the relevant transformer area and importing the total data into hdfs,
(2) monitoring data acquisition by using spark streaming + redis, wherein the monitoring of the acquired data monitors the data at a frequency of every 15 minutes;
step two: data monitoring
(1) Calculating in real time, namely calculating by relying on the scheduling daily electric quantity through a scheduling program, recalculating by relying on the scheduling daily electric quantity through the scheduling program, simultaneously transmitting parameter time into the scheduling program, calculating all daily electric quantities in a window and storing the calculation end in a database according to scheduling statistical time, and updating the latest data of the statistical time into a formal database according to types in the storage process;
(2) recalculating, namely, relying on a scheduling daily electric quantity recalculating program through a scheduling program to calculate daily electric quantities of all the days in a window;
(3) storing the calculation results of real-time calculation and recalculation in a database and updating the calculation results into a formal library, and performing continuous analysis by using a clustering algorithm by using the correlation of the power data change of the single-phase table and the total table of the relevant distribution area to deduce and identify the phase connected with the single-phase table;
(4) monitoring the calculation and storage process by using spark + hbase;
step three: data upload monitoring
(1) And feeding back the data condition to the provincial and municipal metering system in real time, uploading data mainly monitored by the data uploading monitoring system, wherein the data comprise files, index city-level daily electric quantity and index provincial daily electric quantity, and correcting the data problem in time according to the feedback of the provincial and municipal metering system.
In summary, in this embodiment, according to the data monitoring method based on the big power data of this embodiment, the clustering algorithm calculation process is as follows: recording the power change of a single-phase meter at a certain moment and the power change of a certain phase of a station area master table at the same moment, comparing the correlation between the numerical value of the single-phase meter change and the data of the station area master table change, comprehensively analyzing the correlation between the numerical value of the single-phase meter change and the data of the station area master table change for many times, if the low-change correlation or the high-change correlation reaches a threshold value T and the correlation between the data reaches a threshold value P, deducing the phase hung by the single-phase meter, transmitting a parameter time interval from T-3 to T-1 by a scheduling program, analyzing the correlation between the instantaneous power data fluctuation of the electric energy meter of a resident user and the relevant phase power fluctuation of the station area master table by acquiring the instantaneous power data of the electric energy meter in the station area and then distinguishing the phase hung by the electric energy meter installed by the resident user through the edge computing power in a metering terminal or the computing power of a service terminal in a metering automation system, the efficiency and the convenience of identifying the phase position articulated by the single-phase meter are improved, the cost of phase position identification can be greatly saved, meanwhile, the data quality is improved by adopting a clustering algorithm to carry out continuous analysis, the accuracy, timeliness, effectiveness and integrity of data are guaranteed, powerful guarantee is provided for the integration and mining application of the data, the data is monitored in a full-name mode of collection, calculation and uploading, and the accuracy and the timeliness of the data are guaranteed.
The above description is only for the purpose of illustrating the present invention and is not intended to limit the scope of the present invention, and any person skilled in the art can substitute or change the technical solution of the present invention and its conception within the scope of the present invention.
Claims (9)
1. The data monitoring method based on the electric power big data is characterized by comprising the following steps:
the method comprises the following steps: monitoring of acquired data
(1) Acquiring data, acquiring a monophase table and total data of related transformer areas, and importing the collected data into hdfs;
(2) monitoring data acquisition by spark streaming + redis;
step two: data monitoring
(1) Calculating in real time, namely calculating the daily electric quantity dependent on the dispatching through a dispatching program;
(2) recalculating, namely, relying on a scheduling daily electric quantity recalculating program through a scheduling program to calculate daily electric quantities of all the days in a window;
(3) storing the calculation results of real-time calculation and recalculation in a database and updating the calculation results into a formal library;
(4) monitoring the calculation and storage process by using spark + hbase;
step three: data upload monitoring
(1) And feeding back the data condition to the provincial and municipal metering system in real time.
2. The power big data-based data monitoring method of claim 1, wherein in step one, the monitoring of the collected data monitors the data at a frequency of every 15 minutes.
3. The method for monitoring data based on electric power big data as claimed in claim 1, wherein in step two, the correlation between the single-phase table in the database and the power data change of the total table of the relevant station area is continuously analyzed by using a clustering algorithm to deduce and identify the phase hooked by the single-phase table.
4. The method according to claim 1, wherein in the second step, the scheduler relies on a scheduling daily power recalculating program, and the scheduler transmits the parameter time, calculates the daily power of all the days in the window and stores the calculation end in the database according to the scheduling statistical time, and the storing process updates the latest data of the statistical time to the formal database according to the type.
5. The method for monitoring data based on electric power big data as claimed in claim 1, wherein in step three, the data uploaded to monitor the main monitoring data are files, index city level daily electric quantity and index provincial daily electric quantity.
6. The power big data-based data monitoring method according to claim 1, wherein in step three, the data problem is corrected in time according to the feedback of the provincial and urban metering system.
7. The power big data-based data monitoring method according to claim 3, wherein the clustering algorithm is calculated by the following steps: the power change of the single-phase table at a certain moment and the power change of a certain phase of the station area summary table at the same moment are recorded, and then the correlation between the value of the single-phase table change and the data of the station area summary table change is compared.
8. The method according to claim 7, wherein the phase of the single-phase meter is inferred if the low or high correlation reaches a threshold T and the correlation reaches a threshold P by analyzing the correlation between the value of the single-phase meter change and the data of the station area summary table change.
9. The power big data-based data monitoring method according to claim 4, wherein the scheduler incoming parameter time interval is T-3 to T-1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911005332.8A CN110781224A (en) | 2019-10-22 | 2019-10-22 | Data monitoring method based on electric power big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911005332.8A CN110781224A (en) | 2019-10-22 | 2019-10-22 | Data monitoring method based on electric power big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110781224A true CN110781224A (en) | 2020-02-11 |
Family
ID=69384415
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911005332.8A Pending CN110781224A (en) | 2019-10-22 | 2019-10-22 | Data monitoring method based on electric power big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110781224A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116894664A (en) * | 2023-09-11 | 2023-10-17 | 华能吉林发电有限公司 | Data monitoring method based on power big data |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977091A (en) * | 2019-02-25 | 2019-07-05 | 贵州电网有限责任公司 | A kind of distributed computing and storage system |
| CN110082595A (en) * | 2019-06-17 | 2019-08-02 | 鼎信信息科技有限责任公司 | Phase recognition methods, device and the computer equipment of resident's single-phase electric energy meter |
-
2019
- 2019-10-22 CN CN201911005332.8A patent/CN110781224A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977091A (en) * | 2019-02-25 | 2019-07-05 | 贵州电网有限责任公司 | A kind of distributed computing and storage system |
| CN110082595A (en) * | 2019-06-17 | 2019-08-02 | 鼎信信息科技有限责任公司 | Phase recognition methods, device and the computer equipment of resident's single-phase electric energy meter |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116894664A (en) * | 2023-09-11 | 2023-10-17 | 华能吉林发电有限公司 | Data monitoring method based on power big data |
| CN116894664B (en) * | 2023-09-11 | 2023-12-08 | 华能吉林发电有限公司 | Data monitoring method based on power big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111026927B (en) | Low-voltage transformer area running state intelligent monitoring system | |
| CN109683059B (en) | Method and device for determining line loss abnormity, storage medium and processor | |
| CN113659714B (en) | Automatic identification method and system for topological relation of transformer area | |
| CN110928866A (en) | Line topology checking method and system based on power failure information big data analysis | |
| CN113162033B (en) | Automatic identification method and system for topological relation of distribution network station based on frozen current | |
| CN109919328B (en) | Low-voltage branch line power failure research and judgment method based on multi-source power failure event | |
| CN108490285B (en) | Low-voltage transformer area line loss rate calculation method based on voltage drop method | |
| CN112685885A (en) | Transformer area line loss analysis method for comprehensive big data analysis | |
| CN111724060A (en) | Integrated analysis monitoring system and analysis method for transformer area line loss | |
| CN111398859B (en) | User low-voltage cause big data analysis method and system | |
| CN109284837A (en) | A kind of distribution power automation terminal O&M evaluation method and system | |
| CN115940429A (en) | Data acquisition and line loss analysis method based on broadband carrier | |
| CN112615428A (en) | Line loss analysis and treatment system and method | |
| CN109934458B (en) | Power failure real-time analysis method based on minimum power failure common unit | |
| CN103018611A (en) | Non-invasive load monitoring method and system based on current decomposition | |
| CN110781224A (en) | Data monitoring method based on electric power big data | |
| CN112182499B (en) | Low-voltage distribution network topological structure identification method based on time sequence electric quantity data | |
| CN105652157B (en) | Method for analyzing health state of power distribution network based on traveling wave electric quantity | |
| Maaß et al. | Introducing the electrical data recorder as a new capturing device for power grid analysis | |
| CN115995880A (en) | Comprehensive monitoring and analyzing method and system for multidimensional state of power distribution automation terminal | |
| CN115936482A (en) | Low-voltage distribution network power quality analysis method based on power consumption data | |
| CN112202248B (en) | Low-voltage distribution network topology branch refining identification method | |
| CN115603447A (en) | Power utilization monitoring method and system | |
| CN114168662A (en) | Power distribution network problem combing and analyzing method and system based on multiple data sources | |
| CN110727715A (en) | High-reliability-based power consumer daily electric quantity calculation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210730 Address after: 510700 3rd, 4th and 5th floors of building J1 and 3rd floor of building J3, No.11 Kexiang Road, Science City, Luogang District, Guangzhou City, Guangdong Province Applicant after: China South Power Grid International Co.,Ltd. Address before: 510670 No.11 Kexiang Road, Science City, Luogang District, Guangzhou City, Guangdong Province Applicant before: China South Power Grid International Co.,Ltd. Applicant before: POWER GRID TECHNOLOGY RESEARCH CENTER. CHINA SOUTHERN POWER GRID |
|
| TA01 | Transfer of patent application right |