CN110955746B - Electromagnetic data collecting and processing device and method - Google Patents
Electromagnetic data collecting and processing device and method Download PDFInfo
- Publication number
- CN110955746B CN110955746B CN201911008259.XA CN201911008259A CN110955746B CN 110955746 B CN110955746 B CN 110955746B CN 201911008259 A CN201911008259 A CN 201911008259A CN 110955746 B CN110955746 B CN 110955746B
- Authority
- CN
- China
- Prior art keywords
- data
- electromagnetic data
- electromagnetic
- various types
- output
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/165—Combined use of TCP and UDP protocols; selection criteria therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the invention provides an electromagnetic data collecting and processing device and a method, wherein the device comprises: the data input module and the data output module; the data input module is used for collecting various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment; and the data output module is used for carrying out different forms of output processing on the various types of electromagnetic data. The embodiment of the invention uniformly collects and preprocesses various types of electromagnetic data from different sources, and outputs the preprocessed electromagnetic data in different forms, thereby greatly improving the processing capacity of the electromagnetic data.
Description
Technical Field
The invention relates to the technical field of big data collection and processing, in particular to an electromagnetic data collection and processing device and method.
Background
With the development and utilization of electromagnetic space resources, information security and protection issues have attracted increasing attention. Particularly, in the future informatization war, if the electromagnetic space safety construction is not good, serious consequences such as information theft, network damage, paralysis of a command control system, loss of information right and the like can occur.
At present, electromagnetic data is an important basis for analyzing and reflecting electromagnetic space safety. In order to strengthen the safety supervision work of the electromagnetic space, a large number of electromagnetic monitoring devices need to be deployed. The electromagnetic monitoring equipment in the whole electromagnetic space has the characteristics of various types, wide distribution area, dispersed placement positions and the like, so that the electromagnetic data monitored by various types of electromagnetic monitoring equipment are various and huge in quantity.
Therefore, how to efficiently collect various electromagnetic data monitored by various electromagnetic monitoring devices and quickly preprocess the various electromagnetic data becomes a problem to be solved urgently in the field of electromagnetic space security.
Disclosure of Invention
The embodiment of the invention provides an electromagnetic data collecting and processing device and method, which are used for solving the technical problem of how to efficiently collect various types of electromagnetic data monitored by various types of electromagnetic monitoring equipment and quickly preprocess the various types of electromagnetic data.
The embodiment of the invention provides an electromagnetic data collecting and processing device, which comprises: the data input module and the data output module;
the data input module is used for collecting various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment;
and the data output module is used for carrying out different forms of output processing on the various types of electromagnetic data.
Optionally, the various types of electromagnetic data include: electromagnetic data to be transmitted via a TCP protocol, electromagnetic data to be transmitted via a UDP protocol and/or text files.
Optionally, before the data output module is configured to perform different forms of output processing on the various types of electromagnetic data, the electromagnetic data collection processing apparatus further includes: a data preprocessing module;
the data preprocessing module is used for preprocessing the various types of electromagnetic data in the data input module.
Optionally, the preprocessing the various types of electromagnetic data in the data input module includes: performing type conversion, filtering, field addition, field deletion, field modification, canonical matching, and/or normalization operations on the various types of electromagnetic data in the data input module.
Optionally, the performing different forms of output processing on the various types of electromagnetic data includes: and carrying out output processing in different forms on the various types of preprocessed electromagnetic data.
Optionally, the performing different forms of output processing on the preprocessed various types of electromagnetic data includes: and storing the preprocessed various types of electromagnetic data into a database, a text file, a target position and/or message middleware through a TCP or UDP protocol.
The embodiment of the invention provides an electromagnetic data collection processing method, which comprises the following steps:
collecting various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment;
and carrying out different forms of output processing on the various types of electromagnetic data.
Optionally, the various types of electromagnetic data include: electromagnetic data to be transmitted via a TCP protocol, electromagnetic data to be transmitted via a UDP protocol and/or text files.
Optionally, before the different forms of output processing are performed on the various types of electromagnetic data, the electromagnetic data collection processing method further includes: preprocessing the various types of electromagnetic data; wherein the preprocessing the various types of electromagnetic data comprises: performing type conversion, filtering, field addition, field deletion, field modification, canonical matching, and/or normalization operations on the various types of electromagnetic data.
Optionally, the performing different forms of output processing on the various types of electromagnetic data includes: carrying out different forms of output processing on the various types of preprocessed electromagnetic data; wherein the performing different forms of output processing on the preprocessed various types of electromagnetic data comprises: and storing the preprocessed various types of electromagnetic data into a database, a text file, a target position and/or message middleware through a TCP or UDP protocol.
The embodiment of the invention uniformly collects and preprocesses various types of electromagnetic data from different sources, and outputs the preprocessed electromagnetic data in different forms, thereby greatly improving the processing capacity of the electromagnetic data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart illustrating interaction among modules of an electromagnetic data collection and processing apparatus according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating interaction among modules of an electromagnetic data collection and processing apparatus according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating a method for electromagnetic data collection processing according to an embodiment of the invention;
fig. 4 is another schematic flow chart of an electromagnetic data collection processing method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The distributed electromagnetic data collection processing device based on the Logstash provided by the invention has high-efficiency data collection capability. The electromagnetic data monitored by electromagnetic monitoring equipment with large distribution area, dispersed positions and various types can be collected in a centralized manner, and the electromagnetic data can be preprocessed efficiently. In addition, the preprocessed electromagnetic data of the type are output in different forms. The distributed electromagnetic data collecting and processing device based on the Logstash at least comprises: the device comprises a data input module and a data output module.
Fig. 1 is a schematic flowchart illustrating interaction among modules of an electromagnetic data collection and processing apparatus according to an embodiment of the present invention, where the interaction includes: a data input module 11 and a data output module 12.
The data input module 11 is configured to collect various types of electromagnetic data monitored by different types of electromagnetic monitoring devices.
In an embodiment of the invention, different types of electromagnetic monitoring devices are distributed in various areas for monitoring electromagnetic data. The types of electromagnetic data monitored by different types of electromagnetic monitoring equipment are different, and the data volume and the volume of the various types of electromagnetic data monitored are huge. The data input module 11 in the distributed electromagnetic data collection processing device based on the Logstash can collect various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment in a centralized manner.
It should be noted that, in the embodiment of the present invention, each module is connected to the electromagnetic data collection and processing apparatus in a plug-in manner, and different plug-ins can be freely combined according to the requirements and service characteristics of each module. The data input module 11 is connected with the electromagnetic data collection and processing device in a plug-in mode. In an electromagnetic data collection scenario where the amount of data exceeds a first input threshold, the electromagnetic data collection processing apparatus may configure a first input number of data input module plug-ins; in an electromagnetic data collection scenario where the amount of data exceeds a second input threshold, the electromagnetic data collection processing apparatus may configure a second input number of data input module plug-ins; by analogy, in an electromagnetic data collection scenario where the amount of data exceeds an nth input threshold, the electromagnetic data collection processing apparatus may configure an nth input number of data input module plug-ins. The data input module plug-ins in each electromagnetic data collection scenario described above work in parallel to improve the work efficiency of the data input module 11. And load balance of data can be realized among the data input module plug-ins, and the stability and the data processing performance of the electromagnetic data collecting and processing device are greatly enhanced.
The data output module 12 is configured to perform different forms of output processing on the various types of electromagnetic data.
In the embodiment of the present invention, the data output module 12 may adopt different forms of output processing for various types of electromagnetic data monitored by the electromagnetic monitoring device. For example, the various types of electromagnetic data may be stored in a database, in a text file, sent to a target location via TCP or UDP protocols, and/or stored in message middleware.
It should be noted that, in the embodiment of the present invention, the data output module 12 is connected to the electromagnetic data collection and processing device in a plug-in manner. Under the condition that the data volume of the electromagnetic data needing to be output is larger than a first output threshold value, the electromagnetic data collection processing device can be provided with a first output number of data output module plug-ins; under the condition that the data volume of the electromagnetic data needing to be output is larger than a second output threshold value, the electromagnetic data collection processing device can be configured with a second output number of data output module plug-ins; by analogy, in the case that the data volume of the electromagnetic data needing to be output is larger than the nth output threshold, the electromagnetic data collection processing device can configure the nth output number of data output module plug-ins.
The data output module plug-ins work in parallel to improve the work efficiency of the data output module 12. And load balance of data can be realized among the data output module plug-ins, so that the stability and the data processing performance of the electromagnetic data collecting and processing device are greatly enhanced.
The embodiment of the invention collects various types of electromagnetic data from different sources in a centralized manner, outputs and processes various types of electromagnetic data from different sources in different manners, adopts a distributed architecture for the electromagnetic data collecting and processing device, can work in parallel between plug-in units, and enhances the stability and data processing capability of the whole electromagnetic data collecting and processing device.
Further, on the basis of the above apparatus embodiments, the various types of electromagnetic data include: electromagnetic data to be transmitted via a TCP protocol, electromagnetic data to be transmitted via a UDP protocol and/or text files.
In the embodiment of the present invention, for various types of electromagnetic data, such as electromagnetic data that needs to be transmitted via a TCP protocol, electromagnetic data that needs to be transmitted via a UDP protocol, and/or text files, for a device that can communicate with the electromagnetic data collection processing apparatus, the embodiment of the present invention provides a communication protocol, such as TCP, UDP, etc., to complete collection of electromagnetic data from the electromagnetic monitoring device to the data input module 11; for devices that cannot communicate with the electromagnetic data collection processing apparatus, embodiments of the present invention collect electromagnetic data in text form into the data input module 11.
The embodiment of the invention can collect various types of electromagnetic data and improve the data collection capability of the electromagnetic data collection and processing device.
Further, on the basis of the above device embodiment, before the data output module 12 is used to perform different forms of output processing on the various types of electromagnetic data, the electromagnetic data collection processing device further includes: a data preprocessing module 21, as shown in fig. 2; the data preprocessing module 21 is configured to preprocess the various types of electromagnetic data in the data input module 11.
In embodiments of the present invention, the electromagnetic data collected by the data collection module is various types of electromagnetic data from different sources. In order to improve the efficiency of processing data by the electromagnetic data collection processing device, various types of electromagnetic data from different sources need to be preprocessed.
It should be noted that, in the case that the amount of electromagnetic data to be preprocessed is greater than the first preprocessing threshold, the electromagnetic data collection processing apparatus may configure a first preprocessing number of data preprocessing module plug-ins; in the case that the data amount of the electromagnetic data needing to be preprocessed is larger than a second preprocessing threshold value, the electromagnetic data collection processing device can be configured with a second preprocessing number of data preprocessing module plug-ins; by analogy, in the case that the data volume of the electromagnetic data needing to be preprocessed is larger than the Nth preprocessing threshold, the electromagnetic data collection processing device can be configured with the Nth preprocessing number of data preprocessing module plug-ins. The plug-ins of the data preprocessing modules work in parallel to improve the work efficiency of the data preprocessing module 21. And load balance of data can be realized among the data output module plug-ins, so that the stability and the data processing performance of the electromagnetic data collecting and processing device are greatly enhanced.
The embodiment of the invention reduces the data processing pressure of the electromagnetic data collecting and processing device by preprocessing various types of electromagnetic data from different sources; load balance of data can be achieved among the data output module plug-ins, and stability and data processing performance of the electromagnetic data collecting and processing device are greatly enhanced.
Further, on the basis of the above device embodiment, the preprocessing the various types of electromagnetic data in the data input module 11 includes: performing type conversion, filtering, field addition, field deletion, field modification, canonical matching, and/or normalization operations on the various types of electromagnetic data in the data input module 11.
In the embodiment of the present invention, the various types of electromagnetic data in the data input module 11 need to be preprocessed, which specifically includes: the various types of electromagnetic data in the data input module 11 are subjected to type conversion, such as converting character strings into numerical types, filtering, such as removing redundant fields and/or junk fields, field addition, field deletion, field modification, regular matching and/or normalization operations.
It should be noted that the preprocessing of the various types of electromagnetic data in the data input module 11 includes the above-mentioned type conversion, filtering, field addition, field deletion, field modification, regular matching and/or normalization operations, but is not limited to the above-mentioned type conversion, filtering, field addition, field deletion, field modification, regular matching and/or normalization operations.
According to the embodiment of the invention, the pressure of the electromagnetic data collection and processing device for processing data is reduced by preprocessing various types of electromagnetic data from different sources.
Further, on the basis of the above device embodiments, the performing different forms of output processing on the various types of electromagnetic data includes: and carrying out output processing in different forms on the various types of preprocessed electromagnetic data.
In the embodiment of the invention, the preprocessed various types of electromagnetic data have no redundant fields and no garbage fields. The electromagnetic data type is also transformed. Therefore, in order to meet the requirements of users on the use of electromagnetic data, the various types of preprocessed electromagnetic data are output and processed in different forms.
It should be noted that, before the various types of preprocessed electromagnetic data are output in different forms, a data caching module may be further configured to cache the data. This approach addresses the mismatch between data production speed and data consumption speed.
The embodiment of the invention meets users with requirements on electromagnetic data by carrying out different forms of output processing on the various types of preprocessed electromagnetic data.
Further, on the basis of the above device embodiments, the performing, in different forms, output processing on the preprocessed various types of electromagnetic data includes: and storing the preprocessed various types of electromagnetic data into a database, a text file, a target position and/or message middleware through a TCP or UDP protocol.
In embodiments of the present invention, the preprocessed various types of electromagnetic data may be stored in a database such as a structured database and/or an unstructured database, in a text file, sent to a target location via a TCP or UDP protocol, and/or in message middleware such as Redis and/or websocket. Wherein the message middleware is stored mainly for eliminating the difference between the data generation rate and the consumption rate.
According to the embodiment of the invention, the preprocessed various types of electromagnetic data are output and processed in different forms, so that users with requirements on the electromagnetic data can be satisfied.
Fig. 3 is a schematic flow chart of an electromagnetic data collection processing method according to an embodiment of the present invention, including:
and S31, collecting various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment.
In an embodiment of the invention, different types of electromagnetic monitoring devices are distributed in various areas for monitoring electromagnetic data. The types of electromagnetic data monitored by different types of electromagnetic monitoring equipment are different, and the data volume and the volume of the various types of electromagnetic data monitored are huge. The embodiment of the invention can be used for carrying out centralized collection on various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment.
And S32, carrying out output processing in different forms on the various types of electromagnetic data.
In the embodiment of the invention, various types of electromagnetic data monitored by the electromagnetic monitoring equipment are output and processed in different forms. For example, the various types of electromagnetic data may be stored in a database, in a text file, sent to a target location via TCP or UDP protocols, and/or stored in message middleware.
The embodiment of the invention collects various types of electromagnetic data from different sources in a centralized manner, thereby improving the data collection efficiency; the method adopts different output processing modes for various types of electromagnetic data from different sources, and meets users with requirements on the electromagnetic data.
Further, on the basis of the above method embodiment, the various types of electromagnetic data include: electromagnetic data to be transmitted via a TCP protocol, electromagnetic data to be transmitted via a UDP protocol and/or text files.
In the embodiment of the present invention, for various types of electromagnetic data, the embodiment of the present invention provides a communication protocol such as TCP, UDP, or the like, to complete centralized collection of electromagnetic data; the embodiment of the invention also provides the method for collecting the electromagnetic data in a text form in a centralized manner. Accordingly, the various types of electromagnetic data in the embodiments of the present invention include: electromagnetic data to be transmitted via a TCP protocol, electromagnetic data to be transmitted via a UDP protocol and/or text files.
The embodiment of the invention can collect various types of electromagnetic data in a centralized manner, and improves the data collection efficiency.
Further, on the basis of the above method embodiment, before performing output processing in different forms on the various types of electromagnetic data, the electromagnetic data collection processing method further includes: s41, preprocessing the various types of electromagnetic data, as shown in FIG. 4; wherein the preprocessing the various types of electromagnetic data comprises: performing type conversion, filtering, field addition, field deletion, field modification, canonical matching, and/or normalization operations on the various types of electromagnetic data.
In embodiments of the present invention, the collected electromagnetic data is various types of electromagnetic data from different sources. To improve the efficiency of processing data, various types of electromagnetic data from different sources need to be preprocessed. The pretreatment specifically comprises: performing type conversion on the various types of electromagnetic data, such as converting character strings into numerical types, filtering, such as removing redundant fields and/or junk fields, adding fields, deleting fields, modifying fields, performing regular matching and/or normalizing operation.
According to the embodiment of the invention, the data processing pressure is reduced by preprocessing various types of electromagnetic data from different sources.
Further, on the basis of the above method embodiment, the performing different forms of output processing on the various types of electromagnetic data includes: carrying out different forms of output processing on the various types of preprocessed electromagnetic data; wherein the performing different forms of output processing on the preprocessed various types of electromagnetic data comprises: and storing the preprocessed various types of electromagnetic data into a database, a text file, a target position and/or message middleware through a TCP or UDP protocol.
In the embodiment of the invention, the preprocessed various types of electromagnetic data have no redundant fields and no garbage fields. The electromagnetic data type is also transformed. Therefore, in order to meet the requirements of users on the use of electromagnetic data, the various types of preprocessed electromagnetic data are output and processed in different forms. The various types of preprocessed electromagnetic data may be stored in a database such as a structured database and/or an unstructured database, in a text file, sent to a destination location via a TCP or UDP protocol, and/or in a messaging middleware such as Redis and/or websocket. Wherein the message middleware is stored mainly for eliminating the difference between the data generation rate and the consumption rate.
According to the embodiment of the invention, the preprocessed various types of electromagnetic data are output and processed in different forms, so that users with requirements on the electromagnetic data can be satisfied.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (5)
1. An electromagnetic data collection processing apparatus, comprising:
the data input module and the data output module;
the data input module is used for collecting various types of electromagnetic data monitored by different types of electromagnetic monitoring equipment; the various types of electromagnetic data include: electromagnetic data needing to be transmitted by a TCP protocol, electromagnetic data needing to be transmitted by a UDP protocol and/or a text file;
the data input module is connected with the electromagnetic data collecting and processing device in a plug-in mode, and different plug-ins can be freely combined according to the requirements and the service characteristics of each module; in an electromagnetic data collection scenario where the amount of data exceeds a first input threshold, the electromagnetic data collection processing apparatus may configure a first input number of data input module plug-ins; in an electromagnetic data collection scenario where the amount of data exceeds a second input threshold, the electromagnetic data collection processing apparatus may configure a second input number of data input module plug-ins; by analogy, in an electromagnetic data collection scenario in which the data volume exceeds an nth input threshold, the electromagnetic data collection processing apparatus may configure an nth input number of data input module plug-ins;
the data output module is used for carrying out different forms of output processing on the various types of electromagnetic data, and comprises: carrying out different forms of output processing on the various types of preprocessed electromagnetic data;
storing the various types of preprocessed electromagnetic data into a database, storing the various types of preprocessed electromagnetic data into a text file, sending the various types of preprocessed electromagnetic data to a target position through a TCP (transmission control protocol) or UDP (user datagram protocol) protocol and/or storing the various types of preprocessed electromagnetic data into a message middleware;
the data output module is connected with the electromagnetic data collecting and processing device in a plug-in mode; under the condition that the data volume of the electromagnetic data needing to be output is larger than a first output threshold value, the electromagnetic data collection processing device can be provided with a first output number of data output module plug-ins; under the condition that the data volume of the electromagnetic data needing to be output is larger than a second output threshold value, the electromagnetic data collection processing device can be configured with a second output number of data output module plug-ins; by analogy, in the case that the data volume of the electromagnetic data needing to be output is larger than the nth output threshold, the electromagnetic data collection processing device can configure the nth output number of data output module plug-ins.
2. The electromagnetic data collection processing apparatus of claim 1, wherein the data output module is configured to perform different forms of output processing on the various types of electromagnetic data, and further comprises: a data preprocessing module;
the data preprocessing module is used for preprocessing the various types of electromagnetic data in the data input module.
3. The electromagnetic data collection processing apparatus of claim 2, wherein said preprocessing of said various types of electromagnetic data in said data input module comprises: performing type conversion, filtering, field addition, field deletion, field modification, canonical matching, and/or normalization operations on the various types of electromagnetic data in the data input module.
4. An electromagnetic data collection processing method, comprising:
collecting various types of electromagnetic data monitored by different electromagnetic monitoring equipment; the various types of electromagnetic data include: electromagnetic data needing to be transmitted by a TCP protocol, electromagnetic data needing to be transmitted by a UDP protocol and/or a text file;
the data input module is accessed to the electromagnetic data collecting and processing device in a plug-in mode, and different plug-ins can be freely combined according to the requirements and the service characteristics of each module; in an electromagnetic data collection scenario where the amount of data exceeds a first input threshold, the electromagnetic data collection processing apparatus may configure a first input number of data input module plug-ins; in an electromagnetic data collection scenario where the amount of data exceeds a second input threshold, the electromagnetic data collection processing apparatus may configure a second input number of data input module plug-ins; by analogy, in an electromagnetic data collection scenario in which the data volume exceeds an nth input threshold, the electromagnetic data collection processing apparatus may configure an nth input number of data input module plug-ins;
performing different forms of output processing on the various types of electromagnetic data, including: carrying out different forms of output processing on the various types of preprocessed electromagnetic data; storing the various types of preprocessed electromagnetic data into a database, storing the various types of preprocessed electromagnetic data into a text file, sending the various types of preprocessed electromagnetic data to a target position through a TCP (transmission control protocol) or UDP (user datagram protocol) protocol and/or storing the various types of preprocessed electromagnetic data into a message middleware;
the data output module is connected with the electromagnetic data collecting and processing device in a plug-in mode; under the condition that the data volume of the electromagnetic data needing to be output is larger than a first output threshold value, the electromagnetic data collection processing device can be provided with a first output number of data output module plug-ins; under the condition that the data volume of the electromagnetic data needing to be output is larger than a second output threshold value, the electromagnetic data collection processing device can be configured with a second output number of data output module plug-ins; by analogy, in the case that the data volume of the electromagnetic data needing to be output is larger than the nth output threshold, the electromagnetic data collection processing device can configure the nth output number of data output module plug-ins.
5. The electromagnetic data collection processing method according to claim 4, wherein before the different forms of output processing are performed on the various types of electromagnetic data, the electromagnetic data collection processing method further comprises: preprocessing the various types of electromagnetic data; wherein the preprocessing the various types of electromagnetic data comprises: performing type conversion, filtering, field addition, field deletion, field modification, canonical matching, and/or normalization operations on the various types of electromagnetic data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911008259.XA CN110955746B (en) | 2019-10-22 | 2019-10-22 | Electromagnetic data collecting and processing device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911008259.XA CN110955746B (en) | 2019-10-22 | 2019-10-22 | Electromagnetic data collecting and processing device and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110955746A CN110955746A (en) | 2020-04-03 |
| CN110955746B true CN110955746B (en) | 2022-03-22 |
Family
ID=69975741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911008259.XA Active CN110955746B (en) | 2019-10-22 | 2019-10-22 | Electromagnetic data collecting and processing device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110955746B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7805250B2 (en) * | 2006-09-04 | 2010-09-28 | Westerngeco L.L.C. | Methods and apparatus for geophysical exploration via joint inversion |
| CN102109557B (en) * | 2010-12-22 | 2013-04-17 | 东南大学 | Real-time on-line monitoring device for power-frequency electromagnetic data |
| CN104199125A (en) * | 2014-08-29 | 2014-12-10 | 中国石油天然气集团公司 | Earthquake-electromagnetic data acquisition device and earthquake-electromagnetic data acquisition system |
| CN103886269B (en) * | 2014-03-11 | 2017-12-08 | 中国科学院信息工程研究所 | A kind of video information electromagnetic leakage interference prevention method and apparatus |
| CN107992959A (en) * | 2017-04-26 | 2018-05-04 | 国网浙江省电力公司 | A kind of power failure Forecasting Methodology based on electric power big data visualization Neural Network Data digging technology |
| CN108268569A (en) * | 2017-01-04 | 2018-07-10 | 上海宝信软件股份有限公司 | The acquisition of water resource monitoring data and analysis system and method based on big data technology |
| CN108693804A (en) * | 2018-05-11 | 2018-10-23 | 广东冉盛科学仪器有限公司 | A kind of contactless monitoring device and processing method |
| CN106330964B (en) * | 2016-10-14 | 2019-10-11 | 成都信息工程大学 | A kind of network intrusion detection and Initiative Defense linkage control device |
-
2019
- 2019-10-22 CN CN201911008259.XA patent/CN110955746B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7805250B2 (en) * | 2006-09-04 | 2010-09-28 | Westerngeco L.L.C. | Methods and apparatus for geophysical exploration via joint inversion |
| CN102109557B (en) * | 2010-12-22 | 2013-04-17 | 东南大学 | Real-time on-line monitoring device for power-frequency electromagnetic data |
| CN103886269B (en) * | 2014-03-11 | 2017-12-08 | 中国科学院信息工程研究所 | A kind of video information electromagnetic leakage interference prevention method and apparatus |
| CN104199125A (en) * | 2014-08-29 | 2014-12-10 | 中国石油天然气集团公司 | Earthquake-electromagnetic data acquisition device and earthquake-electromagnetic data acquisition system |
| CN106330964B (en) * | 2016-10-14 | 2019-10-11 | 成都信息工程大学 | A kind of network intrusion detection and Initiative Defense linkage control device |
| CN108268569A (en) * | 2017-01-04 | 2018-07-10 | 上海宝信软件股份有限公司 | The acquisition of water resource monitoring data and analysis system and method based on big data technology |
| CN107992959A (en) * | 2017-04-26 | 2018-05-04 | 国网浙江省电力公司 | A kind of power failure Forecasting Methodology based on electric power big data visualization Neural Network Data digging technology |
| CN108693804A (en) * | 2018-05-11 | 2018-10-23 | 广东冉盛科学仪器有限公司 | A kind of contactless monitoring device and processing method |
Non-Patent Citations (4)
| Title |
|---|
| An Efficient Anomaly Detection Framework for Electromagnetic Streaming Data;Degang Sun,Yulan Hu,Zhixin Shi;《ICBDC 2019》;20190512;全文 * |
| An Improved Method for Electromagnetic Streaming Data Anomaly Detection;Degang Sun, Yulan Hu, Zhixin Shi;《Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering》;20190617;全文 * |
| Yanyun Xu ; Jianlin Hu ; Meng Zhang ; Weiqing Huang.Electromagnetic side channel analysis of laser facsimile.《2017 IEEE International Conference on Communications (ICC)》.2017, * |
| 地震监测***的电磁信号的采集设计与实现;庞瑞涛，雍珊珊，王新安，金秀如，曾敬武，韩朝相;《计算机技术与发展》;20180228;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110955746A (en) | 2020-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111885040A (en) | Distributed network situation perception method, system, server and node equipment | |
| CN104331435B (en) | A kind of efficient mass data abstracting method of low influence based on Hadoop big data platforms | |
| JP2022118108A (en) | Log auditing method, device, electronic apparatus, medium and computer program | |
| CN114363042B (en) | Log analysis method, device, equipment and readable storage medium | |
| CN113783845B (en) | Method and device for determining risk level of instance on cloud server, electronic equipment and storage medium | |
| CN111866016A (en) | Log analysis method and system | |
| CN113448812A (en) | Monitoring alarm method and device under micro-service scene | |
| CN110572441A (en) | Ultra-large-scale DPI data processing system and method based on edge calculation | |
| CN111753070A (en) | System and method for processing server monitoring log | |
| WO2021027331A1 (en) | Graph data-based full relationship calculation method and apparatus, device, and storage medium | |
| CN114416685A (en) | Log processing method, system and storage medium | |
| CN114356712A (en) | Data processing method, device, equipment, readable storage medium and program product | |
| CN110955746B (en) | Electromagnetic data collecting and processing device and method | |
| CN115712646A (en) | Alarm strategy generation method, device and storage medium | |
| CN116155541A (en) | Automatic machine learning platform and method for network security application | |
| CN115801395A (en) | Method and device for realizing real-time DDOS (distributed denial of service) situation analysis based on Flink dynamic model | |
| CN115295164A (en) | Medical insurance data processing method and device, electronic equipment and storage medium | |
| CN105162837A (en) | Method and system for improving I/O throughput rate in massive data storage environment | |
| CN113297148B (en) | Method, device and equipment for collecting service log data and readable storage medium | |
| Lee et al. | Malicious traffic compression and classification technique for secure internet of things | |
| CN112612767A (en) | Log file rapid analysis method and device | |
| CN111813542A (en) | Load balancing method and device for parallel processing of large-scale graph analysis tasks | |
| Naukudkar et al. | Enhancing performance of security log analysis using correlation-prediction technique | |
| CN113259878A (en) | Call bill settlement method, system, electronic device and computer readable storage medium | |
| CN115022083B (en) | Abnormal delimitation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |