CN114944957A - Abnormal data detection method and device, computer equipment and storage medium - Google Patents
Abnormal data detection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114944957A CN114944957A CN202210629645.6A CN202210629645A CN114944957A CN 114944957 A CN114944957 A CN 114944957A CN 202210629645 A CN202210629645 A CN 202210629645A CN 114944957 A CN114944957 A CN 114944957A
- Authority
- CN
- China
- Prior art keywords
- data
- uploading
- time
- abnormal
- list
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Computer And Data Communications (AREA)
Abstract
The present invention relates to the field of data processing, and in particular, to a method and an apparatus for detecting abnormal data, a computer device, and a storage medium. Acquiring an RTU data set corresponding to a target remote control system; extracting a corresponding uploading time sequence in each data uploading list; if A is i（j+1） ‑A ij ‑T 0 Entering an abnormal identification link when the current is equal to or more than T; the abnormal recognition link comprises the following steps: if A i（j+1） ‑A ij ‑T 0 ≥∆T s Δ T s When = (j + 1) Δ T, determine the uploading time A i（j+1） Is an abnormal upload time; uploading data to A in the list i（j+1） The corresponding collected data is marked as abnormal data. The invention is in A i（j+1） ‑A ij ‑T 0 When being equal to or greater than T, pass through i（j+1） ‑A ij ‑T 0 ≥∆T s Δ T s If not (j + 1) Δ T, the uploading time of the corresponding acquisition data in the uploading time sequence is identified and then the corresponding acquisition data is uploadedAnd the time is transmitted to identify whether the uploaded collected data is abnormal data or not, and the abnormal data is marked. Therefore, identification and marking of abnormal data in the collected data uploaded by the RTU can be realized.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and an apparatus for detecting abnormal data, a computer device, and a storage medium.
Background
The Modbus protocol is a main data transmission protocol in the industrial field, and because plaintext is adopted for sending when data transmission is carried out, the transmitted data is easily attacked by a network, and great potential safety hazards exist. Therefore, abnormal data exists in the uploaded data. However, in the related art, the efficiency of detecting, calculating and processing the abnormal data in the uploaded data is low, and the abnormal data in the uploaded data cannot be quickly identified.
Disclosure of Invention
In view of the above, the present invention provides an abnormal data detection method, apparatus, computer device and storage medium, which at least partially solve the problems in the prior art.
According to an aspect of the present invention, there is provided an abnormal data detecting method, including:
obtaining an RTU data set U = { B) corresponding to a target remote control system 1 ,B 2 ,…,B w In which B w A data uploading list corresponding to the w-th RTU; the data uploading list comprises corresponding uploading time and collected data when the corresponding RTU uploads data each time, and the collected data in each corresponding data uploading list in the RTU data set have the same attribute;
extract each instituteThe corresponding uploading time sequence C in the data uploading list i =（A i1 ,A i2 ,…,A in ），A ij Uploading time corresponding to jth acquired data in a data uploading list corresponding to the ith RTU;
if A i（j+1） -A ij -T 0 If not, entering abnormal recognition link;
the abnormity identification link comprises the following steps:
if A i（j+1） -A ij -T 0 ≥∆T s Δ T s When = (j + 1) Δ T, determine the uploading time A i（j+1） Is an abnormal upload time;
wherein, T 0 A set time length for the server to poll each RTU in the RTU data set once; Δ T s Is a dynamic threshold;
the patient T is an abnormal threshold which satisfies the following formula: Δ T = T 0 /n*δ；
Wherein n is the number of corresponding RTUs in the RTU data set; delta is a constant, and the value range of delta is [0.8, 1 ];
uploading A in the data uploading list i（j+1） The corresponding collected data is marked as abnormal data.
As a possible implementation manner of the present invention, the abnormality identification step further includes:
obtaining uploading time A i（j+1） The number y of the normal uploading time in the corresponding data uploading list;
when A is i（j+1） -A ij -T 0 ≥∆T s Δ T s If = (j + 1-y) T, determine the corresponding uploading time A in the data uploading list i（j+1） Is an abnormal upload time;
uploading A in the data uploading list i（j+1） The corresponding collected data is marked as abnormal data.
As a possible implementation manner of the present invention, the obtaining of the uploading time a i（j+1） The number y of the normal uploading time in the data uploading list corresponding to the previous time comprises:
if A i -A ij-1 -T 0 Δ T, and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ) When the time is less than or equal to 1, determining the uploading time A i（j+1） Normal upload time;
determining the uploading time A in the data uploading list i（j+1） Number of previous normal upload times.
As a possible implementation manner of the present invention, the abnormality identifying step further includes:
obtaining upload time A i（j+1） The number z of the abnormal uploading time in the corresponding data uploading list;
when A is i（j+1） -A ij -T 0 ≥∆T s Equal and s when z is T, determine the uploading time A i（j+1） Is an abnormal upload time;
uploading A in the data uploading list i（j+1） The corresponding collected data is marked as abnormal data.
As a possible implementation manner of the invention, the acquisition is carried out at the uploading time A i（j+1） The number z of the abnormal uploading time in the data uploading list corresponding to the previous time comprises:
when A is ij -A i（j-1） -T 0 Not less than T and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1 hour, determining the uploading time A i（j+1） Is an abnormal upload time;
determining the uploading time A in the corresponding data uploading list i（j+1） Number of previous exception upload times.
As a possible implementation of the invention, when A ij -A i（j-1） -T 0 Not less than T and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1, determining the corresponding uploading time A in the data uploading list i（j+1） BeforeThe number z of exception upload times includes:
when A is i（j+1） -A ij ≥2T 0 When it is determined
And K is the uploading time A i（j+1） And the uploading time A ij The number of missing flows in between;
determining the uploading time A in the corresponding data uploading list i（j+1） Total number of previous missing traffic;
uploading the corresponding data in the data uploading list at the uploading time A i（j+1） Replacing the sum of the number of the missing flow and the abnormal uploading time with the uploading time A in the corresponding data uploading list i（j+1） The number of previous abnormal upload times z.
As a possible implementation manner of the present invention, the uploading the data to the a in the list i（j+1） The corresponding collected data is marked as abnormal data, and the method comprises the following steps:
the A is added i（j+1） The corresponding collected data mark is at least one mark color; different colors of the marks represent different abnormal conditions.
According to a second aspect of the present invention, there is also provided an abnormal data detecting apparatus comprising:
an obtaining module, configured to obtain an RTU data set U = { B } corresponding to the target remote control system 1 ,B 2 ,…,B w In which B w A data uploading list corresponding to the w-th RTU; the data uploading list comprises corresponding uploading time and collected data when the corresponding RTU uploads data each time, and the collected data in each corresponding data uploading list in the RTU data set have the same attribute;
an extraction module, configured to extract a corresponding upload time sequence C in each of the data upload lists i =（A i1 ,A i2 ,…,A in ），A ij Data upload list corresponding to ith RTUUploading time corresponding to the jth acquired data;
a decision entry module for if A i（j+1） -A ij -T 0 Entering an abnormal identification link when the current is equal to or more than T;
an abnormality determination module for if A i（j+1） -A ij -T 0 ≥∆T s Δ T s When = (j + 1) Δ T, determine the uploading time A i（j+1） An abnormal uploading time;
wherein, T 0 Setting a time length for the server to poll each RTU in the RTU data set once; Δ T s Is a dynamic threshold;
Δ T is an abnormality threshold that satisfies the following formula: Δ T = T 0 /n*δ；
Wherein n is the number of corresponding RTUs in the RTU data set; delta is a constant, and the value range of delta is [0.8, 1 ];
an exception marking module for uploading the data to A in the list i（j+1） The corresponding collected data is marked as abnormal data.
According to a third aspect of the present invention, there is also provided a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the abnormal data detecting method when executing the computer program.
According to a fourth aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the abnormal data detecting method.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
the existing abnormal data identification means generally analyzes the content of each uploaded flow and then identifies and judges according to the content of the uploaded data, and the mode is complex in operation, time-consuming and labor-consuming.
The invention is achieved by the corresponding uploading time characteristic when the RTU uploads the acquired dataAnd (4) quickly identifying abnormal data. Because the Modbus protocol usually has a fixed acquisition period when the data acquisition is performed on the RTU, if the data uploaded by the RTU is not disturbed by the outside world, the data is basically uploaded according to the fixed acquisition period, i.e. T is used 0 Uploading is performed for a period. However, when the system is attacked by abnormal data, the abnormal data usually aims to destroy the normal operation of the network internal device or steal the corresponding data content, so the data volume of the abnormal data is much larger than the data volume of the normal collected data uploaded by the RTU. Therefore, due to the fact that the uploaded data volume is large, uploading time corresponding to abnormal data is delayed correspondingly, and when the data volume of one abnormal data is too large and needs to be divided into a plurality of sub abnormal data to be uploaded, the plurality of sub abnormal data can be continuously uploaded in order to guarantee timeliness of the abnormal data, and therefore the situation that the uploading time of the collected data of a plurality of adjacent RTUs is continuously delayed can be caused.
In A i（j+1） -A ij -T 0 When being equal to or greater than T, pass through i（j+1） -A ij -T 0 ≥∆T s Δ T s And (j + 1) Δ T, and quickly screening abnormal data with serious and continuous delay of the uploading time in the uploading time sequence, further identifying the abnormal data and marking the abnormal data. Because the specific content carried by the flow does not need to be analyzed and analyzed, a large amount of time and labor input can be saved, and the rapid identification and marking of the abnormal data in the collected data uploaded by the RTU can be realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart illustrating a method for detecting abnormal data according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an abnormal data detecting apparatus according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below with reference to the accompanying drawings.
It should be noted that, in the case of no conflict, the features in the following embodiments and examples may be combined with each other; moreover, all other embodiments that can be derived by one of ordinary skill in the art from the embodiments disclosed herein without making any creative effort fall within the scope of the present disclosure.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
Step S10: obtaining an RTU data set U = { B) corresponding to a target remote control system 1 ,B 2 ,...,B w In which B w A data uploading list corresponding to the w-th RTU; the data uploading list comprises corresponding uploading time and collected data when the corresponding RTU uploads data each time, and the collected data in each corresponding data uploading list in the RTU data set have the same attribute;
this embodiment can be applied to the flow monitoring system based on Modbus agreement communication mechanism carries out communication, and this flow monitoring system includes: the monitoring computer carries out communication connection with a plurality of RTUs through the Modbus agreement, and every RTU is connected with a plurality of sensors respectively simultaneously, gathers and uploads the data that every sensor that corresponds gathered through the RTU. Wherein, a plurality of RTUs all are connected with the sensor of gathering same type attribute data. Such as: two RTUs are included in a flow monitoring system, and then both RTUs are connected with a sensor for collecting temperature data.
In addition, each remote control system also corresponds to a database, and the database contains an RTU data set U = { B } of each RTU in the corresponding remote control system 1 ,B 2 ,...,B w }。
Specifically, step S10: the method comprises the steps of obtaining an RTU data set corresponding to a target remote control system, obtaining a data uploading list of a corresponding RTU from data corresponding to the target remote control system, and recording the time of uploading data each time by the RTU and the size of corresponding acquired data in the data uploading list. Taking record content in the data upload list in one time of uploading data as an example, the record content may specifically be: uploading time: 2021/02/25/12: 00: 05, carrying out a reaction; size of the collected data: 2 kb.
Step S20: extracting corresponding uploading time sequence C in each data uploading list i =（A i1 ,A i2 ,...,A in ），A ij Uploading time corresponding to jth acquired data in a data uploading list corresponding to the ith RTU;
specifically, an uploading time sequence in the data uploading list in the corresponding time period may be extracted, and the data uploaded by the RTU in the time period may be subjected to anomaly detection by analyzing the uploading time sequence.
Step S30: if A is i（j+1） -A ij -T 0 Entering an abnormal identification link when the current is equal to or more than T;
the abnormal recognition link comprises the following steps:
step S40: if A is i（j+1） -A ij -T 0 ≥∆T s Δ T s When = (j + 1) Δ T, determine the uploading time A i（j+1） Is an abnormal upload time;
when A is i（j+1） And A ij If the interval exceeds the corresponding value (j + 1), it indicates that A i（j+1） The time after the data uploading is delayed exceeds the set value s . Thus determining A i（j+1） Must be abnormal.
Wherein, T 0 Setting time length for the server to poll each RTU in the RTU data set once; Δ T s Is a dynamic threshold;
Δ T is an abnormality threshold that satisfies the following formula: Δ T = T 0 /n*δ；
Wherein n is the number of corresponding RTUs in the RTU data set; delta is a constant, and the value range of delta is [0.8, 1 ];
in this embodiment, n has a value range of (100- & ltSUB & gt 300- & gt), T 0 The value range of (1) is (100S-200S). Generally, Δ T decreases with increasing n, but Δ T cannot be less than the RTU normal latency range. Therefore, the normal uploading time can be prevented from being judged as the abnormal time by mistake.
Step S50: uploading data to A in the list i（j+1） The corresponding collected data is marked as abnormal data.
According to the step, the collected data corresponding to all abnormal uploading time in the corresponding uploading list are marked as abnormal data, so that abnormal detection of the flow in the corresponding time period is realized.
As a possible embodiment of the present invention, the exception identifying step further includes:
step S41: obtaining uploading time A i（j+1） The number y of the normal uploading time in the corresponding data uploading list;
when A is i（j+1） -A ij -T 0 ≥∆T s Δ T s When = (j + 1-y) Δ T, determine the uploading time A in the corresponding data uploading list i（j+1） Is an abnormal upload time;
step S51: uploading data to A in the list i（j+1） The corresponding collected data is marked as abnormal data.
In this embodiment, y is the acquisition of the upload time a i（j+1） The number of normal uploading time in the corresponding data uploading list. Because the normally uploaded collected data does not delay the uploading time of the subsequent collected data, the time T is equal to s In the calculation process of the value(s), the number of the normally uploaded time should be removed, so that the Δ T can be guaranteed s Does not follow the detected A i（j+1） Has been increasing without limit. If Δ T s If the value of (1) is increased all the time, it will inevitably cause a Δ T s Too large to detect the abnormal condition, therefore Δ T in this embodiment s Calculation of the value, removing A i（j+1） The influence of the acquisition data uploaded at the previous normal uploading time can improve the accuracy of the anomaly detection.
As one possible embodiment of the present invention, step S41: obtaining uploading time A i（j+1） The number y of normal uploading time in the corresponding data uploading list comprises:
step S411: if A i -A ij-1 -T 0 Δ T, and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ) When the time is less than or equal to 1, determining the uploading time A i（j+1） The normal uploading time;
step S412: determining upload time A in data upload list i（j+1） Number of previous normal upload times.
In this example if A i(j+1) -A ij -T 0 < Δ T, then represent A i(j+1) And A ij The interval between is less than the variation threshold, and A can be determined i(j+1) Without delay due to other abnormal conditions, thereby determining the uploading time A i(j+1) Is the normal upload time.
In addition, through (A) i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ) A.ltoreq.1 can be further determined i(j+1) And A ij The previous interval is less than or equal to A ij And A i（j-1） Previous interval, from which A can be stated i(j+1) The situation of delaying occurs without being influenced by abnormal factors, thereby further determining A i(j+1) Is the normal upload time.
As a possible embodiment of the present invention, the exception identifying step further includes:
step S42: obtaining upload time A i（j+1） The number z of the abnormal uploading time in the corresponding data uploading list;
step S52: when A is i（j+1） -A ij -T 0 ≥∆T s Δ T s When z is T, determine the uploading time A i（j+1） Is an abnormal upload time;
step S62: uploading data to A in the list i（j+1） The corresponding collected data is marked as abnormal data.
Δ in this example s Δ T, = z thus s Is only related to the number of abnormal upload times, thereby making Δ T s The value of the time A is more accurate, and simultaneously, a new abnormity judgment method is provided, which can be selected according to the actual situation, when the uploading time A is obtained i（j+1） When the number of the abnormal uploading time in the corresponding data uploading list is more accurate and convenient, the method provided by the embodiment can be used.
As one possible embodiment of the present invention, step S42: obtaining upload time A i（j+1） The number z of the abnormal uploading time in the previous corresponding data uploading list comprises the following steps:
step S421: when A is ij -A i（j-1） -T 0 Not less than T and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1 hour, determining the uploading time A i（j+1） Is an abnormal upload time;
step S422: determining uploading time A in corresponding data uploading list i（j+1） Number of previous exception upload times.
According to this step if A i(j+1) -A ij -T 0 When being equal to T, represent A i(j+1) And A ij The time variation therebetween exceeds a variation threshold, and A i(j+1) And A ij Also satisfies (A) i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1, then represents A i(j-1) 、A ij And A i(j+1) The spacing therebetween is increasingly larger. Due to the fact that under the influence of some abnormal factors, the uploading time of corresponding acquired data is delayed, and therefore the abnormal uploading time can meet the requirement (A) i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1 under the condition of 1. So pass through (A) i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1, A can be determined i(j+1) And A ij Previous space ratio A ij And A i（j-1） The preceding interval being large, i.e. A i(j+1) There is a situation as described above where the upload time is delayed, while A i(j+1) Causes the set variance threshold to be exceeded, and from this, a can be further determined i(j+1) Is an exception upload time.
As one possible embodiment of the present invention, step S421: when A is ij -A i（j-1） -T 0 Not less than T and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1, determining the uploading time A in the corresponding data uploading list i（j+1） The number z of previous exception upload times, including:
step S4211: when A is i（j+1） -A ij ≥2T 0 When it is determined
And K is the uploading time A i（j+1） And upload time A ij The number of missing flows in between;
the missing flow is specifically: and uploading the acquired data missing in the data uploading list. In the actual use process, due to the influence of abnormal factors, the situation that the acquired data of the corresponding uploading time cannot be uploaded occurs, and therefore, relevant records of missing flow cannot exist in the data uploading list.
For example: there is a in the data upload list ij 2021/02/25/12: 00: 00; a. the i(j+1) 2021/02/25/12: 15: 00, and T 0 It is 5 min.
If the collected data are uploaded according to normal conditions, the data are uploaded at A ij And A ij+1 There should also be 2021/02/25/12: 05: 00 and 2021/02/25/12: 10: 00 two upload times. However, due to the influence of the abnormal factor, 2021/02/25/12: 05: 00 and 2021/02/25/12: 10: 00 the collected data corresponding to the two uploading times cannot be uploaded, and further the information corresponding to the collected data is lost in the data uploading list. 2021/02/25/12: 05: 00 and 2021/02/25/12: 10: and 00, the acquired data corresponding to two uploading times is the missing flow.
Step S4212: determining uploading time A in corresponding data uploading list i（j+1） Total number of previous missing traffic;
step S4213: uploading time A in corresponding data uploading list i（j+1） The sum of the previous quantity of the missing flow and the abnormal uploading time is replaced by the uploading time A in the corresponding data uploading list i（j+1） The number of previous abnormal upload times z.
Although the missing traffic is also abnormal data, the missing traffic does not have a corresponding record in the data upload list, so that when the number z of abnormal upload times is determined, the missing traffic is ignored in the conventional method, and thus the finally determined number z of abnormal upload times is inaccurate. In the method in this embodiment, the sum of the number of missing flows and the number of abnormal uploading times is determined as the final z, so that it can be ensured that the calculated z is more accurate, and the accuracy of the detection result of abnormal data in this embodiment is further ensured.
As one possible embodiment of the present invention, step S50: uploading data to A in the list i（j+1） The corresponding collected data is marked as abnormal data, and the method comprises the following steps:
step S501: a is prepared from i（j+1） The corresponding collected data mark is at least one mark color; different colors of the indicia represent different abnormal conditions.
In this embodiment, different colors are set in advance to represent different abnormal conditions, for example, red is represented as abnormal data, yellow is represented as missing flow, and different colors can be used to represent different degrees of abnormality.
According to a second aspect of the present invention, there is also provided an abnormal data detecting apparatus comprising:
an obtaining module, configured to obtain an RTU data set U = { B } corresponding to the target remote control system 1 ,B 2 ,…,B w In which B w A data uploading list corresponding to the w-th RTU; the data uploading list comprises corresponding uploading time and collected data when the corresponding RTU uploads data each time, and the collected data in each corresponding data uploading list in the RTU data set have the same attribute;
an extraction module for extracting the corresponding uploading time sequence C in each data uploading list i =（A i1 ,A i2 ,…,A in ），A ij Uploading time corresponding to jth acquired data in a data uploading list corresponding to the ith RTU;
a decision entry module for if A i（j+1） -A ij -T 0 Entering an abnormal identification link when the current is equal to or more than T;
an abnormality determination module for if A i（j+1） -A ij -T 0 ≥∆T s Δ T s When = (j + 1) Δ T, determine the uploading time A i（j+1） An abnormal uploading time;
wherein, T 0 Setting time length for the server to poll each RTU in the RTU data set once; Δ T s Is a dynamic threshold;
Δ T is an abnormality threshold that satisfies the following formula: Δ T = T 0 /n*δ；
Wherein n is the number of corresponding RTUs in the RTU data set; delta is a constant, and the value range of delta is [0.8, 1 ];
an exception marking module for uploading data to A in the list i（j+1） The corresponding collected data is marked as abnormal data.
According to a third aspect of the present invention, there is also provided a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the abnormal data detection method being implemented when the processor executes the computer program.
According to a fourth aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program, which when executed by a processor implements an abnormal data detecting method.
Moreover, although the steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken into multiple step executions, etc.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device according to this embodiment of the invention. The electronic device is only an example and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
The electronic device is in the form of a general purpose computing device. Components of the electronic device may include, but are not limited to: the at least one processor, the at least one memory, and a bus connecting the various system components (including the memory and the processor).
Wherein the memory stores program code that is executable by the processor to cause the processor to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary method" of the present specification.
The memory may include readable media in the form of volatile memory, such as Random Access Memory (RAM) and/or cache memory, and may further include Read Only Memory (ROM).
The storage may also include a program/utility having a set (at least one) of program modules including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The bus may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.
The electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface. Also, the electronic device may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via a network adapter. The network adapter communicates with other modules of the electronic device over the bus. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the present description, when the program product is run on the terminal device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computing devices (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed, for example, synchronously or asynchronously in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An abnormal data detection method, comprising:
obtaining RTU data set U = { B) corresponding to target remote control system 1 ,B 2 ,…,B w In which B w A data uploading list corresponding to the w-th RTU; the data uploading list comprises corresponding uploading time and collected data when the corresponding RTU uploads data each time, and the collected data in each corresponding data uploading list in the RTU data set have the same attribute;
extracting corresponding uploading time sequence C in each data uploading list i =（A i1 ,A i2 ,…,A in ），A ij Data upload column corresponding to ith RTUUploading time corresponding to jth acquired data in the table;
if A i（j+1） -A ij -T 0 Entering an abnormal identification link when the current is equal to or more than T;
the abnormity identification link comprises the following steps:
if A i（j+1） -A ij -T 0 ≥∆T s Equal and s if Δ (j + 1), the uploading time A is determined i（j+1） Is an abnormal upload time;
wherein, T 0 A set time length for the server to poll each RTU in the RTU data set once; Δ T s Is a dynamic threshold;
Δ T is an abnormality threshold that satisfies the following formula: Δ T = T 0 /n*δ；
Wherein n is the number of corresponding RTUs in the RTU data set; delta is a constant, and the value range of delta is [0.8, 1 ];
uploading A in the data uploading list i（j+1） The corresponding collected data is marked as abnormal data.
2. The abnormal data detecting method according to claim 1, wherein the abnormal recognition step further comprises:
obtaining uploading time A i（j+1） The number y of the normal uploading time in the corresponding data uploading list;
when A is i（j+1） -A ij -T 0 ≥∆T s Equal and s if = (j + 1-y) T, determine the corresponding uploading time A in the data uploading list i（j+1） Is an abnormal upload time;
uploading A in the data uploading list i（j+1） The corresponding collected data is marked as abnormal data.
3. The abnormal data detection method according to claim 2, wherein the acquisition of the upload time a i（j+1） The number y of the normal uploading time in the data uploading list corresponding to the previous time comprises the following steps:
if A i -A ij-1 -T 0 Δ T, and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ) When the time is less than or equal to 1, determining the uploading time A i（j+1） Normal upload time;
determining the upload time A in the data upload list i（j+1） Number of previous normal upload times.
4. The abnormal data detecting method according to claim 1, wherein the abnormal recognition step further comprises:
obtaining upload time A i（j+1） The number z of the abnormal uploading time in the corresponding data uploading list;
when A is i（j+1） -A ij -T 0 ≥∆T s Δ T s When z is T, determine the uploading time A i（j+1） Is an abnormal upload time;
uploading A in the data uploading list i（j+1） The corresponding collected data is marked as abnormal data.
5. The abnormal data detection method according to claim 4, wherein the obtaining at the upload time A i（j+1） The number z of abnormal uploading time in the data uploading list corresponding to the previous time comprises:
when A is ij -A i（j-1） -T 0 Not less than T and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1 hour, determining uploading time A i（j+1） Is an abnormal upload time;
determining the uploading time A in the corresponding data uploading list i（j+1） Number of previous exception upload times.
6. The abnormal data detecting method according to claim 5, wherein when A is reached ij -A i（j-1） -T 0 Not less than T and i(j+1) -A ij -T 0 ）/（A ij -A i（j-1） -T 0 ）>1, determining the corresponding uploading time A in the data uploading list i（j+1） The number of previous exception upload times z includes:
when A is i（j+1） -A ij ≥2T 0 When determining
And K is the upload time A i（j+1） And the uploading time A ij The number of missing flows in between;
determining the uploading time A in the corresponding data uploading list i（j+1） Total number of previous missing traffic;
uploading the corresponding data in the data uploading list at the uploading time A i（j+1） Replacing the sum of the number of the missing flow and the abnormal uploading time with the uploading time A in the corresponding data uploading list i（j+1） The number of previous abnormal upload times z.
7. The abnormal data detection method according to claim 1, wherein the uploading of the data to a in the list is performed i（j+1） The corresponding collected data is marked as abnormal data, and the method comprises the following steps:
the A is added i（j+1） The corresponding collected data mark is at least one mark color; different colors of the marks represent different abnormal conditions.
8. An abnormal data detecting apparatus, comprising:
an obtaining module, configured to obtain an RTU data set U = { B } corresponding to the target remote control system 1 ,B 2 ,…,B w In which B w A data uploading list corresponding to the w-th RTU; the data uploading list comprises corresponding uploading time when the corresponding RTU uploads data each timeAcquiring data, wherein the acquired data in each corresponding data uploading list in the RTU data set have the same attribute;
an extraction module, configured to extract a corresponding upload time sequence C in each of the data upload lists i =（A i1 ,A i2 ,…,A in ），A ij Uploading time corresponding to jth acquired data in a data uploading list corresponding to the ith RTU;
a decision entry module for if A i（j+1） -A ij -T 0 Entering an abnormal identification link when the current is equal to or more than T;
an abnormality determination module for if A i（j+1） -A ij -T 0 ≥∆T s Δ T s If Δ (j + 1), the uploading time A is determined i（j+1） Is an abnormal upload time;
wherein, T 0 Setting a time length for the server to poll each RTU in the RTU data set once; Δ T s Is a dynamic threshold;
Δ T is an abnormality threshold that satisfies the following formula: Δ T = T 0 /n*δ；
Wherein n is the number of corresponding RTUs in the RTU data set; delta is a constant, and the value range of delta is [0.8, 1 ];
an exception marking module for uploading the data to A in the list i（j+1） The corresponding collected data is marked as abnormal data.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the abnormal data detecting method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the abnormal data detecting method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210629645.6A CN114944957B (en) | 2022-06-06 | 2022-06-06 | Abnormal data detection method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210629645.6A CN114944957B (en) | 2022-06-06 | 2022-06-06 | Abnormal data detection method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114944957A true CN114944957A (en) | 2022-08-26 |
| CN114944957B CN114944957B (en) | 2023-01-24 |
Family
ID=82909439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210629645.6A Active CN114944957B (en) | 2022-06-06 | 2022-06-06 | Abnormal data detection method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114944957B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115694844A (en) * | 2023-01-05 | 2023-02-03 | 无锡锐泰节能***科学有限公司 | Internet of things terminal communication method and system based on cloud platform |
| CN116132170A (en) * | 2023-02-13 | 2023-05-16 | 山东云天安全技术有限公司 | Industrial control equipment safety prevention and control system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106502234A (en) * | 2016-10-17 | 2017-03-15 | 重庆邮电大学 | Industrial control system method for detecting abnormality based on double skeleton patterns |
| US20200257608A1 (en) * | 2015-11-19 | 2020-08-13 | Siemens Aktiengesellschaft | Anomaly detection in multiple correlated sensors |
| CN112688946A (en) * | 2020-12-24 | 2021-04-20 | 工业信息安全(四川)创新中心有限公司 | Method, module, storage medium, device and system for constructing abnormality detection features |
| CN114363024A (en) * | 2021-12-22 | 2022-04-15 | 北京六方云信息技术有限公司 | Data encryption transmission method and device, terminal equipment and storage medium |
-
2022
- 2022-06-06 CN CN202210629645.6A patent/CN114944957B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200257608A1 (en) * | 2015-11-19 | 2020-08-13 | Siemens Aktiengesellschaft | Anomaly detection in multiple correlated sensors |
| CN106502234A (en) * | 2016-10-17 | 2017-03-15 | 重庆邮电大学 | Industrial control system method for detecting abnormality based on double skeleton patterns |
| CN112688946A (en) * | 2020-12-24 | 2021-04-20 | 工业信息安全(四川)创新中心有限公司 | Method, module, storage medium, device and system for constructing abnormality detection features |
| CN114363024A (en) * | 2021-12-22 | 2022-04-15 | 北京六方云信息技术有限公司 | Data encryption transmission method and device, terminal equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115694844A (en) * | 2023-01-05 | 2023-02-03 | 无锡锐泰节能***科学有限公司 | Internet of things terminal communication method and system based on cloud platform |
| CN116132170A (en) * | 2023-02-13 | 2023-05-16 | 山东云天安全技术有限公司 | Industrial control equipment safety prevention and control system |
| CN116132170B (en) * | 2023-02-13 | 2023-09-29 | 山东云天安全技术有限公司 | Industrial control equipment safety prevention and control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114944957B (en) | 2023-01-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114944957B (en) | Abnormal data detection method and device, computer equipment and storage medium | |
| JP6669156B2 (en) | Application automatic control system, application automatic control method and program | |
| CN114710369B (en) | Abnormal data detection method and device, computer equipment and storage medium | |
| US7962611B2 (en) | Methods, systems and computer program products for detecting flow-level network traffic anomalies via abstraction levels | |
| US10228994B2 (en) | Information processing system, information processing method, and program | |
| US20180357214A1 (en) | Log analysis system, log analysis method, and storage medium | |
| JP6183450B2 (en) | System analysis apparatus and system analysis method | |
| WO2009110329A1 (en) | Failure analysis device, failure analysis method, and recording medium | |
| CN107209511B (en) | Monitoring control device | |
| WO2017110720A1 (en) | Log analysis system, log analysis method, and recording medium storing program | |
| JP5933386B2 (en) | Data management apparatus and program | |
| WO2018069950A1 (en) | Method, system, and program for analyzing logs | |
| JP2019095822A (en) | Parameter setting method, data analysis device, data analysis system, and program | |
| CN112800061A (en) | Data storage method, device, server and storage medium | |
| CN112639844A (en) | Control system and control method | |
| CN114389881A (en) | Network abnormal flow detection method and device, electronic equipment and storage medium | |
| KR102040371B1 (en) | Apparatus and method for analyzing network attack pattern | |
| US9378082B1 (en) | Diagnosis of storage system component issues via data analytics | |
| JP2008140100A (en) | Information processor, data determination method and program | |
| CN113132393A (en) | Abnormality detection method, abnormality detection device, electronic apparatus, and storage medium | |
| CN117319001A (en) | Network security assessment method, device, storage medium and computer equipment | |
| JP4745881B2 (en) | Network status determination device, network status determination method, and network status determination program | |
| KR20210054725A (en) | Edge based hazard handling device | |
| US8448028B2 (en) | System monitoring method and system monitoring device | |
| US20190294523A1 (en) | Anomaly identification system, method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method, device, computer equipment, and storage medium for detecting abnormal data Effective date of registration: 20230614 Granted publication date: 20230124 Pledgee: Ji'nan rural commercial bank Limited by Share Ltd. high tech branch Pledgor: Shandong Yuntian Safety Technology Co.,Ltd. Registration number: Y2023980043786 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |