CN111177177A - Automatic reporting system and method based on multi-source heterogeneous data fusion - Google Patents
Automatic reporting system and method based on multi-source heterogeneous data fusion Download PDFInfo
- Publication number
- CN111177177A CN111177177A CN201911215664.9A CN201911215664A CN111177177A CN 111177177 A CN111177177 A CN 111177177A CN 201911215664 A CN201911215664 A CN 201911215664A CN 111177177 A CN111177177 A CN 111177177A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- automatic
- fusion
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims description 26
- 230000006870 function Effects 0.000 claims description 28
- 238000013481 data capture Methods 0.000 claims description 14
- 238000002955 isolation Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 claims description 2
- 238000007689 inspection Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims description 2
- 230000005611 electricity Effects 0.000 description 11
- 230000002354 daily effect Effects 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2428—Query predicate definition using graphical user interfaces, including menus and forms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Water Supply & Treatment (AREA)
- Public Health (AREA)
- Marketing (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to an automatic reporting system based on multi-source heterogeneous data fusion, which is characterized in that: the system comprises a global setting and control module, a user-defined table forming a data source, a data capturing module, a data fusion module and an automatic report module. The global setting and control module is a program for setting and controlling the running and operating interface of the automatic reporting system; the data capturing module consists of N data capturing subprograms and is used for capturing data in a uniform XML format of N data source systems; the data fusion module consists of an interface file and N intermediate data files, and can realize automatic fusion of multi-source heterogeneous data; the automatic reporting module is an independent flow automation subprogram, and can read data to be recorded into a reported webpage into an array from the fused interface file and then write the data into a corresponding text box of the webpage, so that the data can be acquired from two or more independent professional systems, fused and then automatically filled into another webpage reporting system.
Description
Technical Field
The invention relates to a computer, in particular to an automatic reporting system and method based on multi-source heterogeneous data fusion.
Background
The power grid dispatching not only needs to organize and coordinate the planned maintenance of the power grid, but also to command and guide the accident handling and abnormal adjustment of the power grid operation. Meanwhile, power grid dispatching operators need to collect and accept or calculate power data of the current day from power plants, substations and large users in jurisdiction every day, and make power reports according to requirements to report to a superior dispatching organization and issue to the interior of an enterprise, the work needs to be continued from 23 o 'clock every night to more than 1 o' clock every morning next day, and when a regular person falls deep, manual operation is easy to cause careless mistakes and mistakes.
In the process of information construction, each department of a power grid enterprise constructs mutually isolated heterogeneous data source systems according to self business and function requirements, and as the development time and the manufacturer of each system are different and do not have uniform planning, development platforms, data structures and database systems are different, professional barriers are prominent, data are not effectively communicated, and a plurality of professional data source systems are formed. Data required by daily electric quantity reports to be made by power grid dispatching operators in the middle and at night every day are stored in the professional data source systems, and if the multi-source heterogeneous data can be fused together and the reports are automatically generated and automatically uploaded to corresponding websites of an upper dispatching department, the error probability of the reported data is greatly reduced, and the working efficiency of the operators is improved.
In the existing multi-source heterogeneous data fusion method, most developers need to be proficient in development tools and platforms of all systems and technical support or open interfaces of original manufacturers, the threshold is high, the entrance is difficult, the existing systems cannot be integrated and utilized, large-scale construction is not dismantled, the system is not reversed, the system is suitable according to the ground, and the system is tried in advance, so that the ubiquitous power internet of things construction principle with the purposes of practicability, economy and convenience is achieved. The multi-source heterogeneous data fusion platform supports personalized setting, is low in threshold, easy to enter a door, strong in robustness and suitable for various professions or industries. Technical support and data provided by each data source system manufacturer are not needed, and only the common inquiry user authority of each system is needed. The method is a multi-source heterogeneous data fusion and automatic reporting method which can integrate and utilize the existing system, does not need to rebuild a data application system, does not need to change the data structure of the original system and has extremely high practical application value.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the defects of the prior art are overcome, and the automatic reporting system and the method thereof based on the fusion of multi-source heterogeneous data are provided.
The general scheme for solving the technical problems of the invention is as follows: an automatic report system based on multi-source heterogeneous data fusion is characterized in that: the system comprises a global setting and control module, a user-defined table forming a data source, a data capturing module, a data fusion module and an automatic reporting module, wherein the global setting and control module is a program for setting and controlling the operation and operation interface of the automatic reporting system; the data capturing module consists of N data capturing subprograms and is used for capturing data in a unified XML format of each data source system at a required moment, wherein N is more than or equal to 2 and is a natural number; the data fusion module is composed of 1 interface file in Excel format and N intermediate data files; the automatic reporting module is an independent flow automation subprogram with an automatic reporting function.
The global setting and control module is used for setting the website, user name, password, automatic starting time of related subprogram, manual automatic switching and switching of foreground and background running modes of each data source system, the data capturing module, the data fusion module and the automatic reporting module, so that a user of the automatic reporting system can modify the contents of the data capturing subprogram of each data source system through each setting item and can manually operate the running of each module through a user interface provided by the module.
The self-defined table for forming the data source is a template file which is well self-defined in advance in each data source system and meets the requirement of a data capturing module and has a unified XML format.
The data capturing subprogram of the data capturing module is an independent function code developed by a Q language and a VBScript script application flow automation technology and is used for capturing a user-defined table with a format which is defined in advance by each data source system at a required moment to form an intermediate data file, and providing data for the data fusion module, wherein the format of the intermediate data file is an Excel format.
The intermediate data file is as follows: the data capture module queries target data according to given query conditions and records in a user-defined set by simulating a manual query step, after the target data are searched, the target data are output in an XML structure, and if the data source system has no option of XML structure output, a query result is converted into the XML structure for output by a query result conversion process; and naming the inquired result as an intermediate data file and storing the intermediate data file under a designated folder in an Excel format.
The data fusion module is composed of 1 interface file in Excel format and N intermediate data files, wherein the interface file comprises 1 data fusion summary table and N intermediate data tables, data of the data fusion summary table is formed by calculating and fusing data of the N intermediate data tables, the structure of the N intermediate data tables is completely the same as the format of the self-defined table of the corresponding data source and the format of the intermediate data file generated by capturing, and the numerical value of the cell corresponding to the intermediate data file is in absolute reference relationship.
The automatic reporting module is an independent flow automation subprogram which is developed by combining Q language and VBScript script with a webpage operation plug-in and has an automatic reporting function, and comprises the steps of initializing an operation program, starting a setting function at regular time, entering a data reporting form page, writing array data into an input text box corresponding to a webpage one by using a webpage operation command, submitting and storing data links until the reported data is checked to be passed.
When the automatic reporting system runs on two or more network computers adopting physical isolation measures, a KVM + encryption USB flash disk selector switch is arranged on the automatic reporting system and is used for quickly transmitting data between the two or more network computers adopting physical isolation measures.
When the automatic reporting system runs on two or more network computers adopting physical isolation measures, the global setting and control module respectively sets the website, the user name, the password, the automatic starting time of related subprograms, the manual automatic switching and the switching content of the foreground and background running modes of each data source system, the data capturing module, the data fusion module and the automatic reporting module on the two or more computers.
The invention solves the technical problem by adopting the specific scheme that: an automatic reporting method based on multi-source heterogeneous data fusion is characterized in that: the method comprises the steps of global setting and control, formation of a user-defined table of a data source, data capture, data fusion and automatic report, and comprises the following steps:
1) global setup and control
the method comprises the steps of setting websites, user names, passwords, related subprogram automatic starting time, manual automatic switching and switching of foreground and background operation modes of each data source system, a data capturing module, a data fusion module and an automatic reporting module;
secondly, providing a user interface for manually operating each module to run;
2) customizing the data source according to the requirements of the data capturing module to form a customized table of the data source, wherein the customized table is a template file with a unified XML format;
3) data capture
the method includes the steps of initializing and setting various environment parameters;
setting a timing function to realize the timing start operation of the subprogram;
starting a browser and opening the data source query system;
fourthly, automatically logging in with the identity of a common user;
⑸, inquiring the target data according to the given inquiry conditions and the records in the custom set;
sixthly, outputting a query result in an XML structure, if the data source system does not have the option, converting the query result into an XML format by using an Office plug-in unit for outputting, and laying a foundation for heterogeneous data fusion;
naming the output query result as a designated intermediate data file name and saving the designated intermediate data file name to a designated folder in an Excel format;
judging whether an intermediate data file of the data source is generated under the appointed folder, if so, finishing the running of the subprogram, and if not, returning the program to start the browser and open the URL of the data source system for rerunning until the data capture is successful;
4) data fusion
the format structures of N intermediate data tables in an interface file and intermediate data files of each data source are completely the same, and the corresponding cell numerical values are in an absolute reference relationship;
⑵, the data of the data fusion summary in the interface file is calculated and fused by a series of formulas from the data of the N intermediate data tables, and the data of the data fusion summary is automatically updated as long as the intermediate data files are replaced;
5) automatic report
starting an automatic reporting program, initializing parameters, and defining an array;
setting a timing function to realize the timing start operation of the report system;
⑶ starting browser and opening the website of report system;
fourthly, automatically logging in with the identity of a common user;
⑸, writing the data of the array into the corresponding positions of the web page one by using the web page operation command;
⑹, submitting and saving data;
the integrity of the validation data is checked for-carriage.
When the automatic reporting system runs on two or more network computers adopting physical isolation measures, a KVM + encryption USB flash disk switch is arranged, interface files are stored on the encryption USB flash disk, and the KVM + encryption USB flash disk switch is used for realizing the rapid data transmission between the computers adopting the forward and reverse physical isolation measures.
The invention has the beneficial effects that:
the data is derived from an application system with two or more independent functions;
the data source systems can be different in manufacturers, different in operating systems, different in databases and different in data structures;
the data collection of each data source system can completely cover all data required by generating a new report, and certain redundancy exists;
fourthly, under the condition that systems of all data sources are not changed and factories are not matched, multi-source heterogeneous data can be effectively integrated, and real fusion of useful data of all systems in the same Excel table after the useful data of all systems are captured is achieved;
the application module supports manual starting and timing automatic starting mode switching setting, supports foreground and background running mode switching setting, and supports convenient setting when the website, the user name and the login password of the data source system are changed.
The automatic reporting system and the method based on the multi-source heterogeneous data fusion are particularly suitable for a scene that the same working role frequently obtains data from two or more independent professional systems and then fuses to generate another professional report, and have the advantages of practicability, economy and convenience of a basic unit, and lead a front-line worker to better use and more wish to use and reduce the burden of basic teams and groups.
Drawings
FIG. 1 is a schematic diagram of an automatic reporting system for a computer according to the present invention;
FIG. 2 is a schematic diagram of a structure of the automatic reporting system for multiple computers according to the present invention;
FIG. 3 is a schematic structural view of embodiment 1;
FIG. 4 is a flowchart of a data fetch subroutine of the data fetch module of embodiment 1;
fig. 5 is a flowchart of the automatic reporting module independent subroutine of embodiment 1.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
Referring to fig. 1 to 5, in embodiment 1, this embodiment is used for a scheduling daily electric quantity report of a power supply system, because the enterprise security protection in this embodiment has the requirement of "horizontal isolation and vertical encryption", and forward and reverse physical isolation must be implemented between a production control large area (including an area i and an area ii) and a management information large area (including an area iii and an area iv), a KVM + encryption usb disk switch needs to be freely switched to implement fast data transfer. In the embodiment, two application programs are adopted, one is operated on the IV-area computer, the other is operated on the II-area computer, and the two computers share one set of keyboard, screen, mouse and encrypted USB flash disk. When the system is started, a keyboard, a screen, a mouse and the encryption U disk are all connected to the computer in the IV area, the data capturing and data fusing functions are completed on the computer in the IV area, and the fused interface file exists in the encryption U disk. And then after the KVM switch is switched, the keyboard, the screen, the mouse and the encrypted USB flash disk are all switched to a computer in the area II, and the automatic report module runs on the computer in the area II. The program firstly reads related data in an interface file in the encrypted USB flash disk into an array, then writes the array data into an input frame corresponding to a reported webpage one by using a webpage operation command, and realizes automatic reporting of the data by using a flow automation program, which comprises the following specific steps:
the automatic reporting system comprises a global setting and control module, a custom table, a data capturing module and a data fusion module, wherein the global setting and control module runs in a computer in an area IV, the custom table forms a data source, and the global setting and control module and the automatic reporting module run in a computer in an area II. The global setting and control module operated by the computer in the IV area is used for globally setting and controlling a user-defined table, a data capturing module and a data fusion module which form a data source; the global setting and control module operated by the computer in the area II is used for globally setting and controlling the automatic report module; the data capturing module consists of three data capturing subprograms and is used for capturing data in a unified XML format of three data source systems at required time; the data fusion module is composed of 1 interface file in Excel format and three intermediate data files; the automatic reporting module is an independent flow automation subprogram with an automatic reporting function.
The global setting and control module operated by the computer in the IV area is used for setting three data source systems of an energy management system (hereinafter referred to as an EMS system), an electric energy acquisition system (hereinafter referred to as an electric acquisition system), a user electricity information acquisition system (hereinafter referred to as an electric acquisition system), a data capture module, a website of the data fusion module, a user name, a password, automatic starting time of related subprograms, manual automatic switching, switching of a foreground and background operation mode and the like, so that a user of the automatic report system can modify the contents of the data capture subprograms of each data source system through the setting items and can manually operate the operation of each module through a user interface provided by the module.
The global setting and control module operated in the computer in the area II is used for setting the website, the user name, the password, the automatic starting time of the automatic report subprogram, the manual automatic switching, the switching of the foreground and background operation modes and the like of a scheduling data reporting system (hereinafter referred to as a 'superior report system'), so that a user of the automatic report system can modify the contents of the automatic report subprogram through the setting items and can manually operate the operation of the automatic report module through a user interface provided by the module.
The three data capturing subprograms of the data capturing module are respectively an EMS system data capturing subprogram, an electric power acquisition system data capturing subprogram and an acquisition system data capturing subprogram. The self-defining functions are inquired by using respective reports of the data source systems in advance, the self-defining tables of the three data source systems are manufactured to form an intermediate data file, data are provided for the data fusion module, and the intermediate data file is in an Excel format.
The user-defined tables of the three data sources are: in an EMS system, a self-defined table of EMS data is defined by using the report service Web query function of Open 3000; in the electric power acquisition system, a user-defined table of electric power acquisition data is defined by utilizing the artificial report function of Com 3000; in the using system, a user-defined table of using data is defined by using a user-defined set and a time-interval electric quantity report query function of the using system, and the user-defined tables of the three data source systems are templates in an XML format. When a data section time period or moment is given, intermediate data files of respective systems are respectively generated after being captured by the data capturing module and are stored under a designated folder in an Excel format.
The interface file of the data fusion module is an Excel file consisting of 1 data fusion summary table and three intermediate data tables; the data in the data fusion summary table is generated by the data of three intermediate data tables of an EMS system, an electric power acquisition system and an acquisition system through correlation operation and fusion; the structures of the three intermediate data tables are completely the same as the format structures of the user-defined tables of the three data sources and the intermediate data files generated by capturing, and the corresponding cell values are in absolute reference relationship.
The flow automation subprogram of the automatic reporting module is an independent subprogram which is developed by combining Q language and VBScript script with a webpage operation plug-in and has an automatic reporting function, and comprises the steps of initializing an operation program, starting a setting function at regular time, entering a data reporting form page, writing array data into an input text box corresponding to a webpage one by using a webpage operation command, submitting and storing the data until the reported data passes the inspection.
The application process of the embodiment is as follows: the method comprises the steps of global setting and control of computer operation in an area IV, formation of a user-defined table of a data source, data capture and data fusion, and also comprises the steps of global setting and control and automatic report of computer operation in an area II, and the method comprises the following steps:
1) global setup and control
the method comprises the steps that an EMS system, an electric sampling system and a website, a user name and a password of the sampling system are arranged on an IV-region computer, and three data capturing subprograms are automatically started for time, manually and automatically switched and switched in a foreground and background operation mode;
providing a user interface for manually operating three data capturing and 1 data fusion module to operate on a computer in an IV area;
setting the website, the user name and the password of the superior reporting system, the automatic starting time of an automatic reporting subprogram, manual automatic switching and switching of a foreground and background running modes on a computer in the area II;
providing a user interface for operating the manual operation automatic report module on a computer in a region II;
2) customizing a data source according to the needs of a data capturing module: in an EMS system, a self-defined table of EMS data is defined by using the report service Web query function of Open 3000; in the electric power acquisition system, a user-defined table of electric power acquisition data is defined by utilizing the artificial report function of Com 3000; in the using system, a user-defined table of using data is defined by using a user-defined set and a time-interval electric quantity report query function of the using system, and the user-defined tables of the three data source systems are templates in an XML format.
3) Data capture
the method comprises the steps of initializing and setting various environment parameters, such as obtaining the maximum horizontal and vertical coordinate values of the operating system and the screen resolution of the current computer;
setting a timing function to realize the timing start operation of the subprogram;
starting a browser and opening the data source query system;
fourthly, automatically logging in with the identity of a common user;
fifthly, after the webpages to be detected are all ready, inquiring target data according to given inquiry conditions and records in the custom set;
sixthly, outputting a query result in an XML structure, and if the data source system does not have the option, converting the query result into an XML format for outputting;
naming the queried result as a custom report, saving the report to a specified folder in an Excel format, and closing a browser;
and judging whether a custom report of the data is generated under the appointed folder or not, if so, finishing the running of the subprogram, if not, returning the program to the URL for starting the browser and opening the data source system for rerun, until the data capture is successful, and after all the data capture is successful, transferring the program to a data fusion module.
The three data capturing modules can capture three intermediate data files through the operation of the 8 steps.
4) Data fusion
the three intermediate data tables and the intermediate data file format structures of the three data sources are completely the same, the corresponding unit cell numerical values are in an absolute reference relationship, and as long as the three intermediate data files are replaced by new data files with the same name and format, the data in the data fusion summary table can be automatically updated;
⑵, the data in the data fusion general table is subjected to relevant operation and fusion by the data of the three intermediate data tables to generate an interface file and store the interface file in the encrypted USB flash disk;
data selection and fusion: in this embodiment, there are three data sources: the electric power collection system can collect the electric power of the company belonging to the transformer substation, partial power plants and individual users; the EMS system can adopt the power and the integral electric quantity of a plurality of users and most power plants belonging to the transformer substation side of a company; the adoption system can adopt the electric quantity of a large user without a self-contained power plant. When the electricity regulation daily report is made, the two intermediate data table structures from the electricity collection system and the EMS system are completely the same, only the data slightly differ, generally, the difference between the two system data at the same point does not exceed 3%, the daily electricity quantity of the same loop or equipment is defaulted to the electricity collection data preferentially, because the electricity collection system is calculated by multiplying the difference of the table code by the multiplying power, the table code is transmitted by a special optical fiber, if no special conditions such as bypass generation, communication interruption and the like exist, the electricity quantity is very accurate, the electricity quantity data of the EMS system adopts the integral electricity quantity of one point per 5 minutes of a daily load curve, the numerical value slightly has an error and can only be used as a backup, namely, the EMS data replaces the electricity collection data when the system judges that the electricity collection data is invalid. The system for using and adopting is also the actual electric quantity calculated by multiplying the difference of the codes by the multiplying factor, but because the communication channel is transmitted by GPRS, the stability of remote areas is poor, the data return is late, and the use and the adoption are specific to large users, and the intersection of the data of the electric collection and the EMS is very small, so that the system can only be independently used as the supplement of part of large user data which is not collected by the other two systems.
Because the EMS system has an automatic bypass function, the electrical production system does not. Therefore, the present embodiment takes the following logical judgment for the trade-off of the electrical acquisition and EMS data: under normal conditions, the electrical data of the electrical production system is used, and once the bypass is generated, the system automatically replaces the electrical production data of the loop or the user with the EMS data.
For the same metering point, the embodiment automatically calculates the maximum value, the minimum value and the average value from the historical data of three years, then multiplies the maximum value by 1.3 to serve as the upper limit, multiplies the minimum value by 0.4 to serve as the lower limit, when the electrical data exceeds the upper limit or the lower limit and power failure is confirmed, EMS data is used for replacing, then EMS data is judged, if the EMS data is confirmed to be invalid, the average value is used for replacing the data and marking, and after the data of the next day is recovered to be normal and real data is obtained, the data of the day is corrected by the difference value between the actual data and the marked data, so that the accumulated value of the electric quantity cannot be wrong.
5) Automatic report
the method comprises the steps of starting an automatic reporting program, initializing and setting parameters, and declaring three arrays of temporarily stored transformer substation, power plant and large user data;
setting a timing function to realize the timing start operation of the report system;
⑶ starting browser and opening the website of report system;
automatically logging in to enter a data transmission form page with the identity of a common user;
writing the data of the array into corresponding positions of the webpage one by using the webpage operation command for the interface file stored in the encrypted USB flash disk;
⑹, submitting and saving data;
and after checking whether the data filled in the webpage are complete or not, closing the browser, if the data are complete, finishing the program, and if the data are incomplete, returning the program to restart the browser, and repeating the process until the reported data are checked to be passed.
The present invention is not limited to the present embodiment, and it is possible for those skilled in the art to easily reproduce and modify the present invention without inventive efforts within the scope of the present invention claimed.
Claims (10)
1. An automatic report system based on multi-source heterogeneous data fusion is characterized in that: the system comprises a global setting and control module, a user-defined table forming a data source, a data capturing module, a data fusion module and an automatic reporting module, wherein the global setting and control module is a program for setting and controlling the operation and operation interface of the automatic reporting system; the data capturing module consists of N data capturing subprograms and is used for capturing data in a unified XML format of each data source system at a required moment, wherein N is more than or equal to 2 and is a natural number; the data fusion module is composed of 1 interface file in Excel format and N intermediate data files; the automatic reporting module is an independent flow automation subprogram with an automatic reporting function.
2. The automatic reporting system based on multi-source heterogeneous data fusion of claim 1, wherein: the automatic reporting system is provided with a KVM and encryption USB flash disk switch and is used for quickly transmitting data between two or more network computers adopting physical isolation measures.
3. The automatic reporting system based on multi-source heterogeneous data fusion of claim 1, wherein: the global setting and control module is used for setting the website, user name, password, automatic starting time of related subprograms, manual automatic switching and switching of foreground and background running modes of each data source system, the data capturing module, the data fusion module and the automatic reporting module, so that a user of the automatic reporting system can modify the contents of the data capturing subprograms of each data source system through each setting item and can manually operate the running of each module through a user interface provided by the module;
when the automatic reporting system runs on two or more network computers adopting physical isolation measures, the global setting and control module respectively sets the website, the user name, the password, the automatic starting time of related subprograms, the manual automatic switching and the switching content of the foreground and background running modes of each data source system, the data capturing module, the data fusion module and the automatic reporting module on the two or more computers.
4. The automatic reporting system based on multi-source heterogeneous data fusion of claim 1, wherein: the self-defined table for forming the data source is a template file which is well self-defined in advance in each data source system and meets the requirement of a data capturing module and has a unified XML format.
5. The automatic reporting system based on multi-source heterogeneous data fusion of claim 1, wherein: the data capturing subprogram of the data capturing module is an independent function code developed by a Q language and a VBScript script application flow automation technology and is used for capturing a user-defined table with a format which is defined in advance by each data source system at a required moment to form an intermediate data file, and providing data for the data fusion module, wherein the format of the intermediate data file is an Excel format.
6. The automatic reporting system based on multi-source heterogeneous data fusion of claim 5, wherein: the intermediate data file is as follows: the data capture module queries target data according to given query conditions and records in a user-defined set by simulating a manual query step, after the target data are searched, the target data are output in an XML structure, and if the data source system has no option of XML structure output, a query result is converted into the XML structure for output by a query result conversion process; and naming the inquired result as an intermediate data file and storing the intermediate data file under a designated folder in an Excel format.
7. The automatic reporting system based on multi-source heterogeneous data fusion of claim 1, wherein: the data fusion module is composed of 1 interface file in Excel format and N intermediate data files, wherein the interface file comprises 1 data fusion summary table and N intermediate data tables, data of the data fusion summary table is formed by calculating and fusing data of the N intermediate data tables, the structure of the N intermediate data tables is completely the same as the format of the self-defined table of the corresponding data source and the format of the intermediate data file generated by capturing, and the numerical value of the cell corresponding to the intermediate data file is in absolute reference relationship.
8. The automatic reporting system based on multi-source heterogeneous data fusion of claim 1, wherein: the automatic reporting module is an independent flow automation subprogram which is developed by combining Q language and VBScript script with a webpage operation plug-in and has an automatic reporting function, and comprises the steps of initializing an operation program, starting a setting function at regular time, entering a data reporting and sending form page, writing array data into an input frame corresponding to a webpage one by using a webpage operation command, and submitting and storing data until the reported data passes inspection.
9. The automatic reporting method based on multi-source heterogeneous data fusion of claim 1, wherein: the method comprises the steps of global setting and control, formation of a user-defined table of a data source, data capture, data fusion and automatic report, and comprises the following steps:
1) global setup and control
the method comprises the steps of setting websites, user names, passwords, related subprogram automatic starting time, manual automatic switching and switching of foreground and background operation modes of each data source system, a data capturing module, a data fusion module and an automatic reporting module;
secondly, providing a user interface for manually operating each module to run;
2) customizing the data source according to the requirements of the data capturing module to form a customized table of the data source, wherein the customized table is a template file with a unified XML format;
3) data capture
the method includes the steps of initializing and setting various environment parameters;
setting a timing function to realize the timing start operation of the subprogram;
starting a browser and opening the data source query system;
fourthly, automatically logging in with the identity of a common user;
⑸, inquiring the target data according to the given inquiry conditions and the records in the custom set;
sixthly, outputting a query result in an XML structure, if the data source system does not have the option, converting the query result into an XML format by using an Office plug-in unit for outputting, and laying a foundation for heterogeneous data fusion;
naming the output query result as a designated intermediate data file name and saving the designated intermediate data file name to a designated folder in an Excel format;
judging whether an intermediate data file of the data source is generated under the appointed folder, if so, finishing the running of the subprogram, and if not, returning the program to start the browser and open the URL of the data source system for rerunning until the data capture is successful;
4) data fusion
the format structures of N intermediate data tables in an interface file and intermediate data files of each data source are completely the same, and the corresponding cell numerical values are in an absolute reference relationship;
⑵, the data of the data fusion summary in the interface file is calculated and fused by a series of formulas from the data of the N intermediate data tables, and the data of the data fusion summary is automatically updated as long as the intermediate data files are replaced;
5) automatic report
starting an automatic reporting program, initializing parameters, and defining an array;
setting a timing function to realize the timing start operation of the report system;
⑶ starting browser and opening the website of report system;
fourthly, automatically logging in with the identity of a common user;
⑸, writing the data of the array into the corresponding positions of the web page one by using the web page operation command;
⑹, submitting and saving data;
the integrity of the validation data is checked for-carriage.
10. The automatic reporting method based on multi-source heterogeneous data fusion of claim 9, wherein: when the automatic reporting system runs in two or more network computers adopting physical isolation measures, a KVM + encryption USB flash disk switch is arranged, interface files are stored in the encryption USB flash disk, and the KVM + encryption USB flash disk switch is used for realizing the rapid data transmission between the computers adopting the forward and reverse physical isolation measures.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911215664.9A CN111177177B (en) | 2019-12-02 | 2019-12-02 | Automatic reporting system based on multi-source heterogeneous data fusion and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911215664.9A CN111177177B (en) | 2019-12-02 | 2019-12-02 | Automatic reporting system based on multi-source heterogeneous data fusion and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111177177A true CN111177177A (en) | 2020-05-19 |
| CN111177177B CN111177177B (en) | 2023-06-16 |
Family
ID=70650097
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911215664.9A Active CN111177177B (en) | 2019-12-02 | 2019-12-02 | Automatic reporting system based on multi-source heterogeneous data fusion and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111177177B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113254244A (en) * | 2021-07-08 | 2021-08-13 | 杭州逍邦网络科技有限公司 | Program fusion system, method and computer device |
| CN113344731A (en) * | 2021-04-02 | 2021-09-03 | 国网吉林省电力有限公司吉林供电公司 | Automatic collection and integration system and method for randomly reporting scattered data |
| CN113642299A (en) * | 2021-07-30 | 2021-11-12 | 国网江苏省电力有限公司南通供电分公司 | One-key generation method based on power grid statistical form |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216817A (en) * | 2007-12-29 | 2008-07-09 | 中国建设银行股份有限公司 | Heterogeneous report form integration and centralized management device and system |
| CN104063761A (en) * | 2014-06-26 | 2014-09-24 | 国家电网公司 | SCADA data development system based on smart grid |
| CN104200402A (en) * | 2014-09-11 | 2014-12-10 | 国家电网公司 | Publishing method and system of source data of multiple data sources in power grid |
| CN106649226A (en) * | 2015-10-29 | 2017-05-10 | 北京国双科技有限公司 | Report generation method and device |
| CN107066499A (en) * | 2016-12-30 | 2017-08-18 | 江苏瑞中数据股份有限公司 | The data query method of multi-source data management and visualization system is stored towards isomery |
| CN110019522A (en) * | 2017-12-01 | 2019-07-16 | 广州明领基因科技有限公司 | Multi-source heterogeneous data integrated system based on XML |
| CN110417873A (en) * | 2019-07-08 | 2019-11-05 | 上海鸿翼软件技术股份有限公司 | A kind of network information extraction system for realizing record webpage interactive operation |
-
2019
- 2019-12-02 CN CN201911215664.9A patent/CN111177177B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216817A (en) * | 2007-12-29 | 2008-07-09 | 中国建设银行股份有限公司 | Heterogeneous report form integration and centralized management device and system |
| CN104063761A (en) * | 2014-06-26 | 2014-09-24 | 国家电网公司 | SCADA data development system based on smart grid |
| CN104200402A (en) * | 2014-09-11 | 2014-12-10 | 国家电网公司 | Publishing method and system of source data of multiple data sources in power grid |
| CN106649226A (en) * | 2015-10-29 | 2017-05-10 | 北京国双科技有限公司 | Report generation method and device |
| CN107066499A (en) * | 2016-12-30 | 2017-08-18 | 江苏瑞中数据股份有限公司 | The data query method of multi-source data management and visualization system is stored towards isomery |
| CN110019522A (en) * | 2017-12-01 | 2019-07-16 | 广州明领基因科技有限公司 | Multi-source heterogeneous data integrated system based on XML |
| CN110417873A (en) * | 2019-07-08 | 2019-11-05 | 上海鸿翼软件技术股份有限公司 | A kind of network information extraction system for realizing record webpage interactive operation |
Non-Patent Citations (5)
| Title |
|---|
| LUIZ F. MARQUES: "Data bases available through APIs using Restify: Characteristics, programming models, and benchmarks" * |
| 周文兵;王波;: "面向服务的集成化自定义表单***研究" * |
| 文静华,张梅,田建强: "基于Excel的企业通用动态报表生成工具的设计" * |
| 潘文博;薛文华;: "自动报表技术在智能电站中的应用" * |
| 黄荔;吕卫强;: "基于数据字典的通用报表处理***" * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113344731A (en) * | 2021-04-02 | 2021-09-03 | 国网吉林省电力有限公司吉林供电公司 | Automatic collection and integration system and method for randomly reporting scattered data |
| CN113344731B (en) * | 2021-04-02 | 2024-04-05 | 国网吉林省电力有限公司吉林供电公司 | Automatic collection and integration system and method for randomly reporting scattered data |
| CN113254244A (en) * | 2021-07-08 | 2021-08-13 | 杭州逍邦网络科技有限公司 | Program fusion system, method and computer device |
| CN113254244B (en) * | 2021-07-08 | 2021-10-19 | 杭州逍邦网络科技有限公司 | Program fusion system, method and computer device |
| CN113642299A (en) * | 2021-07-30 | 2021-11-12 | 国网江苏省电力有限公司南通供电分公司 | One-key generation method based on power grid statistical form |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111177177B (en) | 2023-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177177B (en) | Automatic reporting system based on multi-source heterogeneous data fusion and method thereof | |
| CN106951694B (en) | Adaptive modeling method for online monitoring system of secondary equipment of power system | |
| CN101976235B (en) | Extensible Word report automatically-generating method based on dynamic web page | |
| CN106202447B (en) | A kind of digital cartography system having production and management integration | |
| CN103605705A (en) | SCD (substation configuration description) file management system | |
| CN102884532A (en) | Systems and methods for construction field management and operations with building information modeling | |
| CN102609526A (en) | Internet website content management system | |
| CN108879667B (en) | Power grid closed-loop control power flow simulation method | |
| CN107590749A (en) | A kind of processing method and system with electricity consumption data | |
| CN104063519A (en) | BPA power grid data analyzing and managing method and system based on EXCEL | |
| KR20170122150A (en) | The Monitoring System for Photovoltaic Power Generation and the Method Thereof | |
| KR101843333B1 (en) | The Monitoring System for Photovoltaic Power Generation and the Method Thereof | |
| Varela Souto | Optimization and energy management of a microgrid based on frequency communications | |
| CN103746337A (en) | Relay protection device constant value modifying and checking system | |
| CN105243473A (en) | BI analysis based display method for power marketing decision system data | |
| CN113032651A (en) | Dispatching operation data automatic acquisition and submission management system | |
| CN110659782A (en) | Energy Internet cooperative system control method for improving energy utilization rate | |
| CN108038620A (en) | The comprehensive inspection tour system of substation and inspection method based on cell phone application | |
| CN113377854A (en) | Data integration system based on energy big data | |
| CN104700232A (en) | Standard auditing system for regulation information point table | |
| CN112069400A (en) | Whole-network searching method based on regional power grid information | |
| Soto Gonzalez et al. | FORCE-DISPATCHES Integration-Initial Demonstration | |
| Wang et al. | Global Information Management System for HEPS | |
| CN109001985A (en) | A kind of photovoltaic plant modeling method and device based on coding | |
| Allington et al. | Selected ‘Starter Kit’energy system modelling data for Mauritania (# CCG) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |