CN113515684A

CN113515684A - Abnormal data detection method and device

Info

Publication number: CN113515684A
Application number: CN202010276395.3A
Authority: CN
Inventors: 刘晨; 唐超; 张凯
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-04-09
Filing date: 2020-04-09
Publication date: 2021-10-19
Also published as: GB202011376D0

Abstract

The application discloses an abnormal data detection method, which comprises the following steps: acquiring data to be detected; inputting the data to be detected into a target data reconstruction model, and acquiring target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is a model used for reconstructing the data to be detected according to the characteristic information of the data to be detected and acquiring reconstruction data corresponding to the data to be detected; acquiring reconstruction error information between the data to be detected and the target reconstruction data; and judging whether the data to be detected is abnormal data or not according to the reconstruction error information. According to the method, target reconstruction data corresponding to the data to be detected is obtained through a target data reconstruction model, and after reconstruction error information between the data to be detected and the target reconstruction data is obtained, whether the data to be detected is abnormal data or not can be conveniently and accurately judged according to the reconstruction error information.

Description

Abnormal data detection method and device

Technical Field

The application relates to the technical field of computers, in particular to an abnormal data detection method and device, electronic equipment and storage equipment. The application also relates to a method and a device for obtaining the data reconstruction model, electronic equipment and storage equipment. The application also relates to an abnormal data detection method and device for the network platform, the electronic equipment and the storage equipment. The application also relates to an abnormality detection method and device for the target equipment, the electronic equipment and the storage equipment.

Background

With the continuous development of computer technology, an application program usually generates a large amount of application data during running, for example, a network platform usually generates various user behavior data in the background of the platform for the operation behavior of a user in the platform during running. How to accurately detect abnormal data in the data to quickly solve the problem in the program instruction data causing the abnormal data is increasingly paid attention by people.

Currently, there are two general methods for detecting abnormal data from application data generated by an application program, i.e. from data to be detected: 1. performing threshold-related anomaly detection on certain index data corresponding to a target object, for example, counting historical data of a certain key in a network platform in a unit time according to the number of clicks of the certain key in the network platform or the browsing amount of a certain page in the network platform, and judging whether the real-time data is anomalous or not according to a deviation value of the real-time data and the historical statistical data when performing real-time monitoring; 2. for time series data corresponding to a target object, historical time series data corresponding to the target object is converted into one-dimensional time series data, the one-dimensional time series data is subjected to periodicity and trend line decomposition to obtain a historical period rule or a historical variation trend rule of the one-dimensional time series data, whether the real-time series data is abnormal data or not is judged by analyzing the deviation degree of the real-time series data and the obtained historical period rule or historical variation trend rule, for example, a period variation model of the one-dimensional time series data corresponding to the target object is established through an Autoregressive Integrated Moving Average model (ARIMA), and whether the real-time series data corresponding to the target object is abnormal data or not is judged through the period variation model.

Therefore, in the prior art, one method is only suitable for a single-value analysis scene, namely, only a certain single-value index data is analyzed and detected, but the overall change of the data on a time sequence cannot be analyzed; another method, although it can process time series data, is generally applicable to only one-dimensional time series data, and cannot process time series data having a plurality of dimensions; in addition, in both methods, in the process of training to obtain a model for detecting abnormal data, the difficulty in obtaining abnormal sample data in the training data is high, and the accuracy of the detection result is relatively low. Therefore, when determining whether the data to be detected is abnormal data, the prior art needs to call different models according to the type of the data to be detected, and also has a problem that some types of data cannot be processed, that is, the prior art cannot conveniently and accurately obtain the abnormal data in the data to be detected.

Disclosure of Invention

The embodiment of the application provides an abnormal data detection method, which is used for solving the problem that abnormal data in data to be detected cannot be conveniently and accurately obtained in the prior art.

The embodiment of the application provides an abnormal data detection method, which comprises the following steps: acquiring data to be detected;

inputting the data to be detected into a target data reconstruction model, and acquiring target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is a model used for reconstructing the data to be detected according to the characteristic information of the data to be detected and acquiring reconstruction data corresponding to the data to be detected;

acquiring reconstruction error information between the data to be detected and the target reconstruction data;

and judging whether the data to be detected is abnormal data or not according to the reconstruction error information.

Optionally, the target data reconstruction model is obtained by the following method:

acquiring original sample data, wherein the original sample data comprises at least one of first characteristic type sample data and second characteristic type sample data;

training to obtain the target data reconstruction model according to the original sample data;

the target data reconstruction model comprises an encoding module and a decoding module, wherein the encoding module comprises a first encoding sub-model used for acquiring first sample characteristic information corresponding to the first characteristic type sample data and a second encoding sub-model used for acquiring second sample characteristic information corresponding to the second characteristic type sample data; the decoding module comprises a first decoding submodel and a second decoding submodel, the first decoding submodel is used for reconstructing first reconstruction data corresponding to the first characteristic type sample data according to the first sample characteristic information, and the second decoding submodel is used for reconstructing second reconstruction data corresponding to the second characteristic type sample data according to the second sample characteristic information.

Optionally, the training to obtain the target data reconstruction model according to the original sample data includes:

obtaining a data reconstruction model to be trained, obtaining the first characteristic type sample data from the original sample data, and obtaining the second characteristic type sample data, wherein the data reconstruction model to be trained is a model corresponding to the target data reconstruction model;

training a first coding sub-model in the data reconstruction model to be trained by using the first characteristic type sample data to acquire first sample characteristic information, and training a second coding sub-model in the data reconstruction model to be trained by using the second characteristic type sample data to acquire second sample characteristic information;

training a first decoding submodel in the data reconstruction model to be trained by using the first sample characteristic information to obtain first reconstruction data, and training a second decoding submodel in the data reconstruction model to be trained by using the second sample characteristic information to obtain second reconstruction data;

and adjusting parameters in the data reconstruction model to be trained by acquiring first reconstruction error information corresponding to the first feature type sample data and the first reconstruction data and second reconstruction error information corresponding to the second feature type sample data and the second reconstruction data to obtain the target data reconstruction model meeting a preset convergence condition.

Optionally, the method further includes:

splicing the first sample characteristic information and the second sample characteristic information by using a hidden layer in the data reconstruction model to be trained, and mapping the spliced first sample characteristic information and the spliced second sample characteristic information by using a full-connection layer corresponding to the hidden layer to obtain complete sample characteristic information to be decoded;

the method further comprises the following steps:

and before the second reconstruction data is acquired, acquiring the first sample characteristic information and the second sample characteristic information from the complete sample characteristic information to be decoded.

Optionally, the first feature type sample data includes single-value index data corresponding to a target object, and the second feature type sample data includes time-series feature data corresponding to the target object.

Optionally, the first encoding sub-model and the first decoding sub-model are deep neural network models, and the second encoding sub-model and the second decoding sub-model are cyclic neural network models.

Optionally, the obtaining reconstruction error information between the data to be detected and the target reconstruction data includes:

calculating an error numerical value between the data to be detected and the target reconstruction data;

and obtaining the reconstruction error information according to the error numerical value.

Optionally, the data to be detected includes at least one of data to be detected of a first feature type and data to be detected of a second feature type, and the target reconstruction data includes first target reconstruction data corresponding to the data to be detected of the first feature type and second target reconstruction data corresponding to the data to be detected of the second feature type;

the calculating an error value between the data to be detected and the target reconstruction data includes:

and acquiring a first error numerical value corresponding to the data to be detected of the first characteristic type and the first target reconstruction data, and acquiring a second error numerical value corresponding to the data to be detected of the second characteristic type and the second target reconstruction data.

Optionally, the obtaining a first error value corresponding to the data to be detected of the first feature type and the first target reconstruction data includes:

and obtaining the first error value by calculating a mean square error value between the data to be detected of the first characteristic type and the first target reconstruction data.

Optionally, the obtaining a second error value corresponding to the data to be detected of the second feature type and the second target reconstruction data includes:

and obtaining the second error numerical value by calculating the cross entropy between the data to be detected of the second characteristic type and the second target reconstruction data.

Optionally, the determining, according to the reconstruction error information, whether the data to be detected is abnormal data includes:

and if the reconstruction error information is not less than a preset reconstruction error threshold value, judging that the data to be detected is abnormal data, wherein the preset reconstruction error threshold value is a numerical value obtained in the process of training to obtain the target data reconstruction model.

The embodiment of the present application further provides a method for obtaining a data reconstruction model, including: acquiring original sample data, wherein the original sample data comprises at least one of first characteristic type sample data and second characteristic type sample data;

training to obtain a target data reconstruction model according to the original sample data;

the target data reconstruction model is used for reconstructing the data to be detected according to the characteristic information of the data to be detected and obtaining reconstructed data corresponding to the data to be detected, and comprises an encoding module and a decoding module, wherein the encoding module comprises a first encoding sub-model used for obtaining first sample characteristic information corresponding to the first characteristic type sample data and a second encoding sub-model used for obtaining second sample characteristic information corresponding to the second characteristic type sample data; the decoding module comprises a first decoding submodel and a second decoding submodel, the first decoding submodel is used for reconstructing first reconstruction data corresponding to the first characteristic type sample data according to the first sample characteristic information, and the second decoding submodel is used for reconstructing second reconstruction data corresponding to the second characteristic type sample data according to the second sample characteristic information.

The embodiment of the present application further provides a method for detecting abnormal data for a network platform, including:

acquiring operation data of a platform to be detected corresponding to a target network platform;

inputting the platform operation data to be detected into a target data reconstruction model, and acquiring target platform operation reconstruction data corresponding to the platform operation data to be detected, wherein the target data reconstruction model is used for reconstructing the platform operation data to be detected according to the characteristic information of the platform operation data to be detected and acquiring reconstruction data corresponding to the platform operation data to be detected;

acquiring reconstruction error information between the to-be-detected platform operation data and the target platform operation reconstruction data;

and judging whether the operation data of the platform to be detected is abnormal operation data or not according to the reconstruction error information.

Optionally, if the operation data of the platform to be detected is determined to be abnormal operation data according to the reconstruction error information, the method further includes: acquiring log data corresponding to the abnormal operation data; and obtaining abnormal positioning data according to the log data, wherein the abnormal positioning data is used for positioning program instruction data generating the abnormal operation data.

Optionally, the method further includes: acquiring user information corresponding to the abnormal operation data; and sending abnormal early warning information to user computing equipment corresponding to the user information according to the user information, wherein the abnormal early warning information corresponds to the abnormal operation data, and the user computing equipment is used by a user corresponding to the user information.

Optionally, the method further includes: sending a target service suspension message to the target network platform, wherein the target service suspension message is used for enabling the target network platform to stop receiving the operation request of the user computing device within a preset time range.

Optionally, the target network platform at least includes any one of the following platforms: payment platform, E-commerce platform.

An embodiment of the present application further provides an anomaly detection method for a target device, including:

acquiring to-be-detected running state data of target equipment;

inputting the running state data to be detected into a target data reconstruction model, and acquiring target running state reconstruction data corresponding to the running state data to be detected, wherein the target data reconstruction model is a model used for reconstructing the running state data according to the characteristic information of the running state data and acquiring reconstruction data corresponding to the running state data;

acquiring reconstruction error information between the to-be-detected running state data and the target running state reconstruction data;

and judging whether the target equipment is abnormally operated equipment or not according to the reconstruction error information.

An embodiment of the present application further provides an abnormal data detecting apparatus, including:

the data acquisition unit to be detected is used for acquiring data to be detected;

the target reconstruction data acquisition unit is used for inputting the data to be detected into a target data reconstruction model and acquiring target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is a model used for reconstructing the data to be detected according to the characteristic information of the data to be detected and acquiring reconstruction data corresponding to the data to be detected;

a reconstruction error information obtaining unit, configured to obtain reconstruction error information between the data to be detected and the target reconstruction data;

and the judging unit is used for judging whether the data to be detected is abnormal data or not according to the reconstruction error information.

An embodiment of the present application further provides an electronic device, including:

a processor;

a memory for storing a program of an abnormal data detection method, the apparatus performing the following steps after being powered on and running the program of the abnormal data detection method by the processor:

acquiring data to be detected;

The embodiment of the present application further provides a storage device, in which a program of the abnormal data detection method is stored, where the program is run by a processor and executes the following steps:

acquiring data to be detected;

An embodiment of the present application further provides an obtaining apparatus for a data reconstruction model, including:

the data acquisition unit is used for acquiring original sample data, wherein the original sample data comprises at least one of first characteristic type sample data and second characteristic type sample data;

the training unit is used for training to obtain a target data reconstruction model according to the original sample data;

a processor;

a memory for storing a program of an obtaining method of a data reconstruction model, the apparatus performing the following steps after being powered on and running the program of the obtaining method of the data reconstruction model by the processor:

An embodiment of the present application further provides a storage device, in which a program of an obtaining method of a data reconstruction model is stored, where the program is executed by a processor, and executes the following steps:

An embodiment of the present application further provides an abnormal data detection apparatus for a network platform, including:

the system comprises a to-be-detected platform operation data acquisition unit, a target network platform operation data acquisition unit and a target network platform operation data acquisition unit, wherein the to-be-detected platform operation data acquisition unit is used for acquiring to-be-detected platform operation data corresponding to a target network platform;

the target platform operation reconstruction data acquisition unit is used for inputting the platform operation data to be detected into a target data reconstruction model and acquiring target platform operation reconstruction data corresponding to the platform operation data to be detected, wherein the target data reconstruction model is used for reconstructing the platform operation data to be detected according to the characteristic information of the platform operation data to be detected and acquiring reconstruction data corresponding to the platform access data to be detected;

the reconstruction error information acquisition unit is used for acquiring reconstruction error information between the to-be-detected platform operation data and the target platform operation reconstruction data;

and the abnormal operation data judging unit is used for judging whether the operation data of the platform to be detected is abnormal operation data or not according to the reconstruction error information.

a processor;

a memory for storing a program of an abnormal operation data detection method for a network platform, wherein after the device is powered on and runs the program of the abnormal operation data detection method for the network platform through the processor, the following steps are executed:

The embodiment of the present application further provides a storage device, in which a program of the abnormal data detection method for a network platform is stored, where the program is run by a processor and executes the following steps:

An embodiment of the present application further provides an anomaly detection apparatus for a target device, including:

the device comprises a to-be-detected operating state data acquisition unit, a to-be-detected operating state data acquisition unit and a data processing unit, wherein the to-be-detected operating state data acquisition unit is used for acquiring to-be-detected operating state data of target equipment;

the operation state data reconstruction unit is used for inputting the operation state data to be detected into a target data reconstruction model and acquiring target operation state reconstruction data corresponding to the operation state data to be detected, wherein the target data reconstruction model is a model used for reconstructing the operation state data according to the characteristic information of the operation state data and acquiring reconstruction data corresponding to the operation state data;

the reconstruction error information acquisition unit is used for acquiring reconstruction error information between the to-be-detected running state data and the target running state reconstruction data;

and the judging unit is used for judging whether the target equipment is abnormally operated equipment or not according to the reconstruction error information.

a processor;

a memory for storing a program of an abnormality detection method for a target device, which executes the following steps after the device is powered on and the program of the abnormality detection method for the target device is executed by the processor:

acquiring to-be-detected running state data of target equipment;

An embodiment of the present application further provides a storage device, in which a program of an abnormality detection method for a target device is stored, where the program is run by a processor and executes the following steps:

acquiring to-be-detected running state data of target equipment;

Compared with the prior art, the method has the following advantages:

the embodiment of the application provides an abnormal data detection method, which comprises the following steps: acquiring data to be detected; inputting the data to be detected into a target data reconstruction model, and acquiring target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is a model used for reconstructing the data to be detected according to the characteristic information of the data to be detected and acquiring reconstruction data corresponding to the data to be detected; acquiring reconstruction error information between the data to be detected and the target reconstruction data; and judging whether the data to be detected is abnormal data or not according to the reconstruction error information. When judging whether the data to be detected is abnormal data, the method does not need to pay attention to the specific type of the data, only needs to obtain target reconstruction data corresponding to the data to be detected through a target data reconstruction model, and can conveniently and accurately judge whether the data to be detected is abnormal data by obtaining reconstruction error information between the data to be detected and the target reconstruction data.

The embodiment of the present application further provides a method for obtaining a data reconstruction model, including: acquiring original sample data, wherein the original sample data comprises at least one of first characteristic type sample data and second characteristic type sample data; training to obtain a target data reconstruction model according to the original sample data; the target data reconstruction model is used for reconstructing the data to be detected according to the characteristic information of the data to be detected and obtaining reconstructed data corresponding to the data to be detected, and comprises an encoding module and a decoding module, wherein the encoding module comprises a first encoding sub-model used for obtaining first sample characteristic information corresponding to the first characteristic type sample data and a second encoding sub-model used for obtaining second sample characteristic information corresponding to the second characteristic type sample data; the decoding module comprises a first decoding submodel and a second decoding submodel, the first decoding submodel is used for reconstructing first reconstruction data corresponding to the first characteristic type sample data according to the first sample characteristic information, and the second decoding submodel is used for reconstructing second reconstruction data corresponding to the second characteristic type sample data according to the second sample characteristic information. According to the method, the sub models corresponding to the sample data with different characteristic types in the original sample data are set, the reconstruction data corresponding to the sample data with different characteristic types are adaptively reconstructed, and the reconstruction data corresponding to the data to be detected can be conveniently obtained.

The embodiment of the present application further provides a method for detecting abnormal data for a network platform, including: acquiring operation data of a platform to be detected corresponding to a target network platform; inputting the platform operation data to be detected into a target data reconstruction model, and acquiring target platform operation reconstruction data corresponding to the platform operation data to be detected, wherein the target data reconstruction model is used for reconstructing the platform operation data to be detected according to the characteristic information of the platform operation data to be detected and acquiring reconstruction data corresponding to the platform operation data to be detected; obtaining reconstruction error information between the to-be-detected platform operation data and the target platform operation reconstruction data; and judging whether the operation data of the platform to be detected is abnormal operation data or not according to the reconstruction error information. The method can conveniently and accurately judge whether the operation data of the platform to be detected corresponding to the target network platform is abnormal operation data or not, so that the computing equipment corresponding to the target network platform can perform quick response processing aiming at the abnormal access data.

An embodiment of the present application further provides an anomaly detection method for a target device, including: acquiring to-be-detected running state data of target equipment; inputting the running state data to be detected into a target data reconstruction model, and acquiring target running state reconstruction data corresponding to the running state data to be detected, wherein the target data reconstruction model is a model used for reconstructing the running state data according to the characteristic information of the running state data and acquiring reconstruction data corresponding to the running state data; acquiring reconstruction error information between the to-be-detected running state data and the target running state reconstruction data; and judging whether the target equipment is abnormally operated equipment or not according to the reconstruction error information. The method can conveniently and accurately judge whether the target equipment is abnormally operated by judging whether the to-be-detected operation state data of the target equipment is abnormal data.

Drawings

Fig. 1 is a schematic view of an application scenario of an abnormal data detection method according to a first embodiment of the present application.

Fig. 2 is a flowchart of an abnormal data detection method according to a first embodiment of the present application.

Fig. 3 is a data processing diagram of an abnormal data detection method according to a first embodiment of the present application.

Fig. 4 is a flowchart of a method for obtaining a data reconstruction model according to a second embodiment of the present application.

Fig. 5 is a flowchart of an abnormal data detection method for a network platform according to a third embodiment of the present application.

Fig. 6 is a schematic diagram of an abnormal data detecting apparatus according to a fourth embodiment of the present application.

Fig. 7 is a schematic diagram of an electronic device according to a fifth embodiment of the present application.

Fig. 8 is a schematic diagram of an apparatus for obtaining a data reconstruction model according to a seventh embodiment of the present application.

Fig. 9 is a schematic diagram of an abnormal data detection apparatus for a network platform according to a tenth embodiment of the present application.

Fig. 10 is a flowchart of an abnormality detection method for a target device according to a thirteenth embodiment of the present application.

Fig. 11 is a schematic diagram of an abnormality detection method for a target device according to a fourteenth embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

In order to make those skilled in the art better understand the scheme of the present application, a detailed description is given below of a specific application scenario of an embodiment of the abnormal data detection method provided in the present application. The abnormal data detection method provided in the first embodiment of the present application may be applied to a scenario in which the abnormal data detection method interacts with a client and a server, as shown in fig. 1, which is an application scenario diagram of the abnormal data detection method provided in the first embodiment of the present application.

In specific implementation, the method may be for enabling the computing device to quickly, conveniently and accurately obtain abnormal data in the data to be detected, and further enabling a user or the computing device to respond to program instruction data generating the abnormal data in time, after the client obtains the data to be detected, the client sends the data to be detected to the server, where the data to be detected may be platform operation data for a certain network platform, or may also be running state data of a certain machine device, or may also be detection data of a certain entity object acquired by a certain computing device after obtaining authorization; the server side obtains the data to be detected sent by the client side, inputs the data to be detected into a target data reconstruction model, and obtains target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is used for reconstructing the data to be detected according to the characteristic information of the data to be detected and obtaining reconstruction data corresponding to the data to be detected; then, the server side obtains reconstruction error information between the data to be detected and the target reconstruction data, and judges whether the data to be detected is abnormal data or not according to the reconstruction error information; then, sending the judgment result to the client; of course, if the data to be detected is abnormal data, the server may send the determination result to the client, and send abnormal location data for locating the program instruction data generating the abnormal data to the user according to the difference of the abnormal data.

Of course, the method can also be applied to the client or the server separately. For example, after obtaining authorization, the server may obtain data to be detected from data, such as a user behavior log or a system log, generated by the server when providing service to the user according to the application program; then, inputting the data to be detected into a target data reconstruction model, acquiring target reconstruction data corresponding to the data to be detected, and judging whether the data to be detected is abnormal data or not by acquiring reconstruction error information between the data to be detected and the target reconstruction data; if the data to be detected is judged to be abnormal data, the server side can push the abnormal data to the user computing equipment corresponding to the abnormal data, or can also acquire abnormal positioning data corresponding to the abnormal data and used for positioning program instruction data generating the abnormal data, and push the abnormal positioning data to the user computing equipment corresponding to the abnormal data, or the server side can also generate abnormal early warning information corresponding to the abnormal data and push the abnormal early warning information to the user computing equipment corresponding to the abnormal data, and if the server side judges that the abnormal data is abnormal intrusion data, a target service pause message can be sent to the client side, wherein the target service pause message is used for enabling the client side to stop receiving operation requests of the user computing equipment corresponding to the abnormal data within a preset time range, the user computing device is a computing device used by a user corresponding to the user information.

The client may be a mobile terminal device, such as a mobile phone, a tablet computer, or the like, or may be a commonly used computer device. The server is generally a server, and the server may be a physical server or a cloud server, and is not particularly limited herein.

It should be noted that the above application scenarios are only specific examples of the abnormal data detection method provided in the first embodiment of the present application, and the above application scenarios are provided for facilitating understanding of the method and are not intended to limit the method.

Fig. 2 is a flowchart of an abnormal data detection method according to a first embodiment of the present application. The method provided by the first embodiment of the present application is described below with reference to fig. 2.

Step S201, data to be detected is acquired.

The data to be detected refers to data corresponding to a target object, and the data may be attribute data corresponding to the target object, specifically, may be single-value index data corresponding to the target object, or may be time-series characteristic data corresponding to the target object, where the target object generally refers to an entity or a virtual object.

For example, for an intelligent wearable device, such as an intelligent watch or an intelligent bracelet, the corresponding data to be detected may be a heart rate value detected by the intelligent wearable device within a certain time range; for a vehicle object, the corresponding data to be detected can be relative position information of the vehicle object and surrounding vehicles in time sequence when the vehicle object is driven; for a certain network platform, such as a payment platform, an e-commerce platform, and the like, the data to be detected may be user operation data of a user in the network platform after authorization, such as click behavior or click times for a certain key, browsing times for a certain page, or operation time corresponding to the network platform being subjected to an intrusion operation initiated by the user using a computing device; for a certain device, such as a network device like a router, a switch, etc., the data to be detected may be operation state data of the device, such as data like network throughput in unit time.

It should be noted that, in the first embodiment of the present application, the data to be detected includes at least one of data to be detected of the first feature type and data to be detected of the second feature type; the data to be detected of the first characteristic type can be single-value index data corresponding to a target object; the data to be detected of the second characteristic type can be time sequence characteristic data corresponding to the target object; in addition, the time series characteristic data may be one-dimensional time series characteristic data, that is, certain single-valued index data in time series corresponding to the target object; the time-series feature data may be multi-dimensional time-series feature data, that is, single-value index data of a plurality of dimensions in time series corresponding to the target object. For example, the one-dimensional time series characteristic data corresponding to the smart wearable device may be in the form of (time, heart rate value); the multidimensional timing characteristic data corresponding to the network platform may be in the form of (time, number of certain key clicks, number of certain page views, number of users, number of IPs …).

Step S202, inputting the data to be detected into a target data reconstruction model, and acquiring target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is a model used for reconstructing the data to be detected according to the characteristic information of the data to be detected and acquiring reconstruction data corresponding to the data to be detected.

In the prior art, when anomaly detection needs to be performed on data to be detected, an anomaly detection model obtained through pre-training is usually used to determine whether the data to be detected is anomalous data, for example, the anomaly detection model is used to detect whether the data to be detected of a single-value index type is anomalous data. However, this method faces a problem that it is generally difficult to obtain abnormal sample data that can be used for supervised learning when training to obtain an abnormal detection model, that is, in practice, normal sample data corresponding to data to be detected is extremely easy to obtain, and abnormal sample data is relatively a small probability event, so abnormal sample data is usually extremely few and is not easy to obtain. For example, when the intelligent wearable device is used for heart rate detection, or when the time sequence relative position information corresponding to the vehicle object is used for accident responsibility determination, or when the platform operation data corresponding to the network platform is used for judging whether intrusion operation aiming at the network platform exists or whether abnormal network blockage exists, the normal heart rate value, the time sequence relative position information during normal driving and the platform operation data under normal conditions can be generally acquired in a large batch, and the acquisition difficulty of the corresponding abnormal data is relatively high and the quantity of the corresponding abnormal data is very small. Therefore, the anomaly detection model obtained by using a large amount of normal sample data and a small amount of anomaly sample data is time-consuming and labor-consuming in the acquisition process, and has the problem of low detection accuracy.

For the problems in the prior art, the abnormal data detection method provided in the first embodiment of the present application obtains target reconstruction data corresponding to data to be detected through a target data reconstruction model obtained through pre-training, and determines whether the data to be detected is abnormal data by obtaining reconstruction error information between the data to be detected and the target reconstruction data.

The target data reconstruction model is used for reconstructing the data to be detected according to the characteristic information of the data to be detected and obtaining reconstructed data corresponding to the data to be detected; in addition, for the problem that the anomaly detection model in the prior art can only process data of a certain feature type only and cannot process data of different feature types at the same time when detecting data to be detected, that is, for the problem that the anomaly detection model in the prior art can only process single-value index data or can only convert multidimensional time sequence feature data into one-dimensional time sequence feature data for processing, the target data reconstruction model provided in the first embodiment of the present application includes at least two submodels in order to increase the learning capability of the model, and the two submodels can be processed in a self-adaptive manner according to the type of the input data to be detected.

The target data reconstruction model may be obtained by: acquiring original sample data, wherein the original sample data comprises at least one of first characteristic type sample data and second characteristic type sample data; training to obtain the target data reconstruction model according to the original sample data; the target data reconstruction model comprises an encoding module and a decoding module, wherein the encoding module comprises a first encoding sub-model used for acquiring first sample characteristic information corresponding to the first characteristic type sample data and a second encoding sub-model used for acquiring second sample characteristic information corresponding to the second characteristic type sample data; the decoding module comprises a first decoding submodel and a second decoding submodel, the first decoding submodel is used for reconstructing first reconstruction data corresponding to the first characteristic type sample data according to the first sample characteristic information, and the second decoding submodel is used for reconstructing second reconstruction data corresponding to the second characteristic type sample data according to the second sample characteristic information.

As shown in fig. 3, which is a data processing schematic diagram of an abnormal data detection method provided in the first embodiment of the present application, it can be known from fig. 3 that a structure of a target data reconstruction model provided in the first embodiment of the present application may be a model structure corresponding to an Auto Encoder (AE) framework, that is, the target data reconstruction model completes reconstruction of data to be detected through an encoding module and a decoding module included in the target data reconstruction model.

It should be noted that, in the first embodiment of the present application, the first feature type sample data is sample data corresponding to the data to be detected of the first feature type in step S201, that is, the first feature type sample data is single-valued index data corresponding to the target object; the second feature type sample data is sample data corresponding to the data to be detected of the second feature type in step S201, that is, the second feature type sample data is time sequence feature data corresponding to the target object, and may specifically be one-dimensional time sequence feature data or multidimensional time sequence feature data. In addition, the first encoding sub-model and the first decoding sub-model in the target data reconstruction model may be a Deep Neural Network model (DNN), and the second encoding sub-model and the second decoding sub-model in the target data reconstruction model may be a Recurrent Neural Network model (RNN). Of course, in specific implementation, the target data reconstruction model may also be set to have other structures, and is not particularly limited herein.

In addition, the training to obtain the target data reconstruction model according to the original sample data includes: obtaining a data reconstruction model to be trained, obtaining the first characteristic type sample data from the original sample data, and obtaining the second characteristic type sample data, wherein the data reconstruction model to be trained is a model corresponding to the target data reconstruction model; training a first coding sub-model in the data reconstruction model to be trained by using the first characteristic type sample data to acquire first sample characteristic information, and training a second coding sub-model in the data reconstruction model to be trained by using the second characteristic type sample data to acquire second sample characteristic information; training a first decoding submodel in the data reconstruction model to be trained by using the first sample characteristic information to obtain first reconstruction data, and training a second decoding submodel in the data reconstruction model to be trained by using the second sample characteristic information to obtain second reconstruction data; and adjusting parameters in the data reconstruction model to be trained by acquiring first reconstruction error information corresponding to the first feature type sample data and the first reconstruction data and second reconstruction error information corresponding to the second feature type sample data and the second reconstruction data to obtain the target data reconstruction model meeting a preset convergence condition. It should be noted that the preset convergence condition may be set according to actual situations, and is not particularly limited herein.

In addition, in order to maintain the integrity of the model, so as to conveniently train the model and improve the accuracy of the processing result of the model, the data reconstruction model to be trained further includes a hidden layer and a fully connected layer corresponding to the hidden layer, and in the process of obtaining the target data reconstruction model by training the data reconstruction model to be trained by using original sample data, the method further includes: splicing the first sample characteristic information and the second sample characteristic information by using a hidden layer in the data reconstruction model to be trained, and mapping the spliced first sample characteristic information and the spliced second sample characteristic information by using a full-connection layer corresponding to the hidden layer to obtain complete sample characteristic information to be decoded; the method further comprises the following steps: and before the second reconstruction data is acquired, acquiring the first sample characteristic information and the second sample characteristic information from the complete sample characteristic information to be decoded.

For example, for a target data reconstruction model used for reconstructing reconstruction data of platform operation data to be detected corresponding to a network platform, in the process of obtaining the target data reconstruction model through training, historical platform operation data may be obtained from a server corresponding to the network platform as original sample data for training the model, and the target platform operation reconstruction data corresponding to the platform operation data to be detected is obtained through training through the sample data of at least the two feature types included in the original sample data.

It should be further noted that, in the first embodiment of the present application, whether the data to be detected is abnormal data is determined by obtaining reconstruction error information between the data to be detected and the target reconstruction data, specifically because: in the process of training to obtain a target data reconstruction model, normal sample data is easy to obtain, so that the normal sample data can be well reconstructed, namely, the error between the obtained reconstructed data and the normal sample data is usually small; however, since the abnormal sample data is difficult to obtain, the abnormal sample data is generally difficult to reconstruct, and therefore, an error between the obtained reconstructed data and the abnormal sample data is generally large; therefore, the original sample data, such as historical data corresponding to the data to be detected, is used for training the data reconstruction model to be trained, and then the target data reconstruction model with smaller reconstruction error between the reconstructed data and the original sample data is obtained, so that the data to be detected can be reconstructed better when the data to be detected is normal data, and the error between the reconstructed data and the data to be detected is larger when the data to be detected is abnormal data, and the accuracy for judging whether the data to be detected is abnormal data can be improved by the method.

In the above, the method for obtaining the target data reconstruction model according to the first embodiment of the present application and the reason why the target data reconstruction model is used to obtain the reconstruction data corresponding to the data to be detected are introduced in detail, and it can be known from the above description that great efforts are not required to label the abnormal data in the original sample data when the target data reconstruction model is obtained, so that the method can relatively simply obtain the target data reconstruction model, and further conveniently detect the abnormality of the data to be detected.

After step S202, step S203 is executed to acquire reconstruction error information between the data to be detected and the target reconstruction data.

As can be seen from the above description, the obtaining of the reconstruction error information between the data to be detected and the target reconstruction data, which is an error value between the data to be detected and the target reconstruction data, includes: calculating an error numerical value between the data to be detected and the target reconstruction data; and obtaining the reconstruction error information according to the error numerical value.

The data to be detected comprises at least one of data to be detected of a first characteristic type and data to be detected of a second characteristic type, and the corresponding target reconstruction data corresponding to the data to be detected also comprises first target reconstruction data corresponding to the data to be detected of the first characteristic type and second target reconstruction data corresponding to the data to be detected of the second characteristic type. Therefore, the calculating the error value between the data to be detected and the target reconstruction data includes: and acquiring a first error numerical value corresponding to the data to be detected of the first characteristic type and the first target reconstruction data, and acquiring a second error numerical value corresponding to the data to be detected of the second characteristic type and the second target reconstruction data.

The obtaining of the first error value corresponding to the data to be detected of the first feature type and the first target reconstruction data includes: and obtaining the first error value by calculating a mean square error value between the data to be detected of the first characteristic type and the first target reconstruction data.

For example, the data to be detected for the first feature type corresponding to the target object is in the form of (entity ID, feature 1, feature 2), such as (ID00001, 0.331, 0.8716); the first target reconstruction data corresponding to this data is (id00001, 0.231, 0.8826); then, the mean square error value between the data to be detected of the first feature type and the first target reconstruction data is ((0.331-0.231) ^2+ (0.8716-0.8826) ^2) ^ 0.5.

The obtaining of the second error value corresponding to the data to be detected of the second feature type and the second target reconstruction data includes: and obtaining the second error numerical value by calculating the cross entropy between the data to be detected of the second characteristic type and the second target reconstruction data.

For example, the data to be detected for the second feature type corresponding to the target object is in the form of (entity ID, time series feature 1, time series feature 2), such as (ID00001, 2020-1-2003: 20, 0.1, 0.4); the second target reconstruction data corresponding to this data is (id00001, 2020-1-2003: 20, 0.15, 0.32); then, the cross entropy between the data to be detected of the second feature type and the second target reconstruction data is (0.1 × log (0.15) + (1-0.1) × log (1-0.15) +0.4 × log (0.32) + (1-0.4) × log (1-0.32))/2.

In the above, how to obtain the reconstruction error information between the data to be detected and the target reconstruction data is described in detail. It should be noted that, in specific implementation, the reconstruction error information may also be obtained by other methods, which is not described herein again.

And step S204, judging whether the data to be detected is abnormal data or not according to the reconstruction error information.

After the reconstruction error information, such as an error value, between the data to be detected and the target reconstruction data is obtained in step S203, whether the data to be detected is abnormal data or not can be determined according to the reconstruction error information.

The determining whether the data to be detected is abnormal data according to the reconstruction error information includes: and if the reconstruction error information is not less than a preset reconstruction error threshold, determining that the data to be detected is abnormal data, wherein the preset reconstruction error threshold is a numerical value obtained in the process of obtaining the target data reconstruction model through training, and the reconstruction error threshold can be obtained according to the error change trend between the obtained reconstruction data and the original sample data in the process of obtaining the target data reconstruction model through training, which is not described herein again.

In addition, when the target object is a machine device, such as a router, a switch, or other network device, after obtaining the to-be-detected operating state data and the operating state reconstruction data of the network device, when it is determined that the to-be-detected operating state data is abnormal data according to reconstruction error information between the to-be-detected operating state data and the operating state reconstruction data, it may be determined that the network device is an abnormally-operating device, and at this time, the network device may be restarted or abnormal early warning information may be pushed to the user computing device, so that the network device may be restored to normal operation as soon as possible.

In addition, when the target object is a certain network platform, such as a payment platform or an e-commerce platform, because the platform generally relates to sensitive data, when reconstruction error information between the two data is obtained through platform operation data corresponding to the network platform and corresponding platform operation reconstruction data, and the platform operation data is judged to be abnormal operation data according to the reconstruction error information, such as intrusion data or risk transaction data aiming at the network platform, user information corresponding to the abnormal operation data can be obtained after authorization is obtained, and abnormal early warning information is sent to user computing equipment corresponding to the user information; meanwhile, a service suspension message for stopping the network platform from receiving the operation request of the user computing device within a preset time range can be sent to the network platform, so that the network platform is prevented from further intrusion attack.

It should be noted that, in specific implementation, the abnormal data monitoring method described in this application may also be applied to other scenarios as needed, and details are not described here.

In summary, the abnormal data detection method provided in the first embodiment of the present application includes: acquiring data to be detected; inputting the data to be detected into a target data reconstruction model, and acquiring target reconstruction data corresponding to the data to be detected, wherein the target data reconstruction model is a model used for reconstructing the data to be detected according to the characteristic information of the data to be detected and acquiring reconstruction data corresponding to the data to be detected; acquiring reconstruction error information between the data to be detected and the target reconstruction data; and judging whether the data to be detected is abnormal data or not according to the reconstruction error information. When judging whether the data to be detected is abnormal data, the method does not need to pay attention to the specific type of the data, only needs to obtain target reconstruction data corresponding to the data to be detected through a target data reconstruction model, and can conveniently and accurately judge whether the data to be detected is abnormal data by obtaining reconstruction error information between the data to be detected and the target reconstruction data.

In correspondence with the abnormal data detection method provided in the first embodiment of the present application, the second embodiment of the present application further provides a method for obtaining a data reconstruction model, please refer to fig. 4, which is a flowchart of the method for obtaining a data reconstruction model provided in the second embodiment of the present application, wherein some steps have been described in detail in the first embodiment of the present application, so that the description herein is relatively simple, and for the relevant points, reference may be made to some descriptions in the method provided in the first embodiment of the present application, and the processing procedures described below are only exemplary.

Step S401, obtaining original sample data, wherein the original sample data includes at least one of first characteristic type sample data and second characteristic type sample data.

Step S402, training to obtain a target data reconstruction model according to the original sample data;

Optionally, the training to obtain the target data reconstruction model according to the original sample data includes: obtaining a data reconstruction model to be trained, obtaining the first characteristic type sample data from the original sample data, and obtaining the second characteristic type sample data, wherein the data reconstruction model to be trained is a model corresponding to the target data reconstruction model; training a first coding sub-model in the data reconstruction model to be trained by using the first characteristic type sample data to acquire first sample characteristic information, and training a second coding sub-model in the data reconstruction model to be trained by using the second characteristic type sample data to acquire second sample characteristic information; training a first decoding submodel in the data reconstruction model to be trained by using the first sample characteristic information to obtain first reconstruction data, and training a second decoding submodel in the data reconstruction model to be trained by using the second sample characteristic information to obtain second reconstruction data; and adjusting parameters in the data reconstruction model to be trained by acquiring first reconstruction error information corresponding to the first feature type sample data and the first reconstruction data and second reconstruction error information corresponding to the second feature type sample data and the second reconstruction data to obtain the target data reconstruction model meeting a preset convergence condition.

Optionally, the method further includes: splicing the first sample characteristic information and the second sample characteristic information by using a hidden layer in the data reconstruction model to be trained, and mapping the spliced first sample characteristic information and the spliced second sample characteristic information by using a full-connection layer corresponding to the hidden layer to obtain complete sample characteristic information to be decoded; the method further comprises the following steps: and before the second reconstruction data is acquired, acquiring the first sample characteristic information and the second sample characteristic information from the complete sample characteristic information to be decoded.

Corresponding to the abnormal data detection method provided in the first embodiment of the present application, a third embodiment of the present application further provides an abnormal access data detection method for a network platform, please refer to fig. 5, which is a flowchart of the abnormal access data detection method for a network platform provided in the third embodiment of the present application, wherein some steps have been described in detail in the first embodiment of the present application, so that the description herein is relatively simple, and for the relevant points, reference may be made to some descriptions in the method provided in the first embodiment of the present application, and the processing procedure described below is only exemplary.

Step S501, obtaining operation data of the platform to be detected corresponding to the target network platform.

Step S502, inputting the platform operation data to be detected into a target data reconstruction model, and acquiring target platform operation reconstruction data corresponding to the platform operation data to be detected, wherein the target data reconstruction model is used for reconstructing the platform operation data to be detected according to the characteristic information of the platform operation data to be detected and acquiring reconstruction data corresponding to the platform operation data to be detected.

Step S503, obtaining reconstruction error information between the to-be-detected platform operation data and the target platform operation reconstruction data.

Step S504, according to the reconstruction error information, judging whether the operation data of the platform to be detected is abnormal operation data.

In correspondence with the abnormal data detection method provided in the first embodiment of the present application, a fourth embodiment of the present application further provides an abnormal data detection apparatus, please refer to fig. 6, which is a schematic diagram of the abnormal data detection apparatus provided in the fourth embodiment of the present application. An abnormal data detection apparatus provided in a fourth embodiment of the present application includes:

the to-be-detected data acquiring unit 601 is configured to acquire to-be-detected data.

A target reconstruction data obtaining unit 602, configured to input the data to be detected into a target data reconstruction model, and obtain target reconstruction data corresponding to the data to be detected, where the target data reconstruction model is a model that is used to reconstruct the data to be detected according to the feature information of the data to be detected and obtain reconstruction data corresponding to the data to be detected.

A reconstruction error information obtaining unit 603, configured to obtain reconstruction error information between the data to be detected and the target reconstruction data.

A determining unit 604, configured to determine whether the data to be detected is abnormal data according to the reconstruction error information.

Optionally, the reconstruction error information obtaining unit is specifically configured to: calculating an error numerical value between the data to be detected and the target reconstruction data; and obtaining the reconstruction error information according to the error numerical value.

Optionally, the data to be detected includes at least one of data to be detected of a first feature type and data to be detected of a second feature type, and the target reconstruction data includes first target reconstruction data corresponding to the data to be detected of the first feature type and second target reconstruction data corresponding to the data to be detected of the second feature type; the calculating an error value between the data to be detected and the target reconstruction data includes: and acquiring a first error numerical value corresponding to the data to be detected of the first characteristic type and the first target reconstruction data, and acquiring a second error numerical value corresponding to the data to be detected of the second characteristic type and the second target reconstruction data.

Optionally, the obtaining a first error value corresponding to the data to be detected of the first feature type and the first target reconstruction data includes: and obtaining the first error value by calculating a mean square error value between the data to be detected of the first characteristic type and the first target reconstruction data.

Optionally, the obtaining a second error value corresponding to the data to be detected of the second feature type and the second target reconstruction data includes: and obtaining the second error numerical value by calculating the cross entropy between the data to be detected of the second characteristic type and the second target reconstruction data.

Optionally, the determining, according to the reconstruction error information, whether the data to be detected is abnormal data includes: and if the reconstruction error information is not less than a preset reconstruction error threshold value, judging that the data to be detected is abnormal data, wherein the preset reconstruction error threshold value is a numerical value obtained in the process of training to obtain the target data reconstruction model.

Corresponding to the method for detecting abnormal data provided by the first embodiment of the present application, a fifth embodiment of the present application further provides an electronic device, please refer to fig. 7, which is a schematic diagram of an electronic device provided by the fifth embodiment of the present application. A fifth embodiment of the present application provides an electronic device including:

a processor 701;

a memory 702 for storing a program of an abnormal data detection method, wherein after the apparatus is powered on and the program of the abnormal data detection method is executed by the processor, the following steps are performed:

acquiring data to be detected;

In correspondence with the abnormal data detection method provided in the first embodiment of the present application, a sixth embodiment of the present application further provides a storage device, since the embodiment of the storage device is substantially similar to the embodiment of the method, the description is relatively simple, and the relevant points can be referred to the partial description of the embodiment of the method, and the embodiment of the storage device described below is only illustrative.

A storage device according to a sixth embodiment of the present application stores a program of an abnormal data detection method, where the program is executed by a processor to perform the following steps:

acquiring data to be detected;

Corresponding to the method for obtaining a data reconstruction model provided in the second embodiment of the present application, a seventh embodiment of the present application further provides a device for obtaining a data reconstruction model, please refer to fig. 7, which is a schematic diagram of the device for obtaining a data reconstruction model provided in the seventh embodiment of the present application. A seventh embodiment of the present application provides an apparatus for obtaining a data reconstruction model, including:

a data obtaining unit 701, configured to obtain original sample data, where the original sample data includes at least one of first feature type sample data and second feature type sample data.

A training unit 702, configured to train to obtain a target data reconstruction model according to the original sample data;

Corresponding to the method for obtaining the data reconstruction model provided in the second embodiment of the present application, an eighth embodiment of the present application further provides an electronic device, which is substantially similar to the method embodiment, so that the description is relatively simple, and for relevant points, reference may be made to part of the description of the method embodiment, and the electronic device embodiment described below is only illustrative. An eighth embodiment of the present application provides an electronic device including:

a processor;

Corresponding to the method for obtaining the data reconstruction model provided in the second embodiment of the present application, a ninth embodiment of the present application further provides a storage device, since the embodiment of the storage device is substantially similar to the embodiment of the method, the description is relatively simple, and for relevant points, reference may be made to part of the description of the embodiment of the method, and the embodiment of the storage device described below is only illustrative.

A storage device according to a ninth embodiment of the present application stores a program of a method for obtaining a data reconstruction model, the program being executed by a processor and performing the steps of:

Corresponding to the method for detecting abnormal data of a network platform provided in the third embodiment of the present application, a tenth embodiment of the present application further provides an apparatus for detecting abnormal data of a network platform, please refer to fig. 9, which is a schematic diagram of the apparatus for detecting abnormal data of a network platform provided in the tenth embodiment of the present application. An abnormal access data detection device for a network platform provided by a tenth embodiment of the present application includes the following components:

the to-be-detected platform operation data acquiring unit 901 is configured to acquire to-be-detected platform operation data corresponding to the target network platform.

The target platform operation reconstruction data obtaining unit 902 is configured to input the platform operation data to be detected into a target data reconstruction model, and obtain target platform operation reconstruction data corresponding to the platform operation data to be detected, where the target data reconstruction model is configured to reconstruct the platform operation data to be detected according to the feature information of the platform operation data to be detected, and obtain reconstruction data corresponding to the platform operation data to be detected.

A reconstruction error information obtaining unit 903, configured to obtain reconstruction error information between the to-be-detected platform operation data and the target platform operation reconstruction data.

An abnormal operation data determining unit 904, configured to determine whether the platform operation data to be detected is abnormal operation data according to the reconstruction error information.

Corresponding to the method for detecting abnormal data of a network platform provided in the third embodiment of the present application, the eleventh embodiment of the present application further provides an electronic device, since the embodiment of the electronic device is substantially similar to the embodiment of the method, the description is relatively simple, and for relevant points, reference may be made to part of the description of the embodiment of the method, and the embodiment of the electronic device described below is only illustrative. An electronic device provided in an eleventh embodiment of the present application includes:

a processor;

a memory for storing a program of an abnormal data detection method for a network platform, wherein after the device is powered on and runs the program of the abnormal data detection method for the network platform through the processor, the following steps are executed:

Corresponding to the method for detecting abnormal data of a network platform provided in the third embodiment of the present application, the twelfth embodiment of the present application further provides a storage device, since the embodiment of the storage device is substantially similar to the embodiment of the method, the description is relatively simple, and for relevant points, reference may be made to part of the description of the embodiment of the method, and the embodiment of the storage device described below is only illustrative.

A storage device according to a twelfth embodiment of the present application stores a program of an abnormal data detection method for a network platform, where the program is executed by a processor to perform the following steps:

In correspondence with the method for detecting abnormal data provided in the first embodiment of the present application, a thirteenth embodiment of the present application further provides a method for detecting an abnormality of a target device, please refer to fig. 10, which is a flowchart of the method for detecting an abnormality of a target device provided in the thirteenth embodiment of the present application, wherein some steps have been described in detail in the first embodiment of the present application, so that the description herein is relatively simple, and for the relevant points, reference may be made to some descriptions in the method provided in the first embodiment of the present application, and the processing procedure described below is only exemplary.

Step S1001, obtains the to-be-detected operating state data of the target device.

Step S1002, inputting the operation state data to be detected into a target data reconstruction model, and acquiring target operation state reconstruction data corresponding to the operation state data to be detected, where the target data reconstruction model is a model used to reconstruct the operation state data according to characteristic information of the operation state data and acquire reconstruction data corresponding to the operation state data.

Step S1003, obtaining reconstruction error information between the to-be-detected operating state data and the target operating state reconstruction data.

Step S1004, determining whether the target device is an abnormally operating device according to the reconstruction error information.

Corresponding to the method for detecting an abnormality of a target device provided in the thirteenth embodiment of the present application, a fourteenth embodiment of the present application further provides an apparatus for detecting an abnormality of a target device, please refer to fig. 11, which is a schematic diagram of the apparatus for detecting an abnormality of a target device provided in the fourteenth embodiment of the present application. A fourteenth embodiment of the present application provides an abnormality detection apparatus for a target device, including:

the operation state data to be detected acquiring unit 1101 is configured to acquire operation state data to be detected of the target device.

The operation state data reconstruction unit 1102 is configured to input the operation state data to be detected into a target data reconstruction model, and acquire target operation state reconstruction data corresponding to the operation state data to be detected, where the target data reconstruction model is a model used to reconstruct the operation state data according to characteristic information of the operation state data and acquire reconstruction data corresponding to the operation state data.

A reconstruction error information obtaining unit 1103, configured to obtain reconstruction error information between the to-be-detected operating state data and the target operating state reconstruction data.

A determining unit 1104, configured to determine whether the target device is an abnormally operating device according to the reconstruction error information.

Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims

1. An abnormal data detection method, comprising:

acquiring data to be detected;

2. The abnormal data monitoring method according to claim 1, wherein the target data reconstruction model is obtained by:

3. The abnormal data monitoring method according to claim 2, wherein the training to obtain the target data reconstruction model according to the original sample data comprises:

4. The abnormal data detecting method according to claim 3, further comprising:

the method further comprises the following steps:

5. The abnormal data detection method according to claim 2, wherein the first feature type sample data includes single-valued index data corresponding to a target object, and the second feature type sample data includes time-series feature data corresponding to the target object.

6. The abnormal data detection method of claim 2, wherein the first encoding sub-model and the first decoding sub-model are deep neural network models, and the second encoding sub-model and the second decoding sub-model are recurrent neural network models.

7. The abnormal data detection model of claim 1, wherein the obtaining of reconstruction error information between the data to be detected and the target reconstruction data comprises:

8. The abnormal data detection method according to claim 7, wherein the data to be detected includes at least one of data to be detected of a first feature type and data to be detected of a second feature type, and the target reconstruction data includes first target reconstruction data corresponding to the data to be detected of the first feature type and second target reconstruction data corresponding to the data to be detected of the second feature type;

9. The abnormal data detection method according to claim 8, wherein the obtaining a first error value corresponding to the data to be detected of the first feature type and the first target reconstruction data includes:

10. The abnormal data detection method according to claim 8, wherein the obtaining a second error value corresponding to the data to be detected of the second feature type and the second target reconstruction data includes:

11. The abnormal data detection method according to claim 1, wherein the determining whether the data to be detected is abnormal data according to the reconstruction error information includes:

12. A method for obtaining a data reconstruction model, comprising:

13. An abnormal data detection method for a network platform is characterized by comprising the following steps:

inputting the platform operation data to be detected into a target data reconstruction model, and acquiring target platform operation reconstruction data corresponding to the platform operation data to be detected, wherein the target data reconstruction model is a model used for reconstructing the platform operation data to be detected according to the characteristic information of the platform operation data to be detected and acquiring reconstruction data corresponding to the platform operation data to be detected;

acquiring reconstruction error information between the to-be-detected platform operation data and the target reconstruction platform operation data;

14. The method according to claim 13, wherein if the operation data of the platform to be detected is determined to be abnormal operation data according to the reconstruction error information, the method further comprises:

acquiring log data corresponding to the abnormal operation data;

and obtaining abnormal positioning data according to the log data, wherein the abnormal positioning data is used for positioning program instruction data generating the abnormal operation data.

15. The abnormal data detection method for network platforms according to claim 13, further comprising:

acquiring user information corresponding to the abnormal operation data;

and sending abnormal early warning information to user computing equipment corresponding to the user information according to the user information, wherein the abnormal early warning information corresponds to the abnormal operation data, and the user computing equipment is used by a user corresponding to the user information.

16. The abnormal data detection method for network platforms according to claim 15, further comprising:

sending a target service suspension message to the target network platform, wherein the target service suspension message is used for enabling the target network platform to stop receiving the operation request of the user computing device within a preset time range.

17. The abnormal data detection method for network platforms according to claim 13, wherein the target network platform includes at least any one of the following platforms: payment platform, E-commerce platform.

18. An abnormality detection method for a target device, characterized by comprising:

acquiring to-be-detected running state data of target equipment;

19. An abnormal data detecting apparatus, comprising:

20. An electronic device, comprising:

a processor;

acquiring data to be detected;

21. A storage device, in which a program of an abnormal data detecting method is stored, the program being executed by a processor and performing the steps of:

acquiring data to be detected;

22. An abnormal data detecting apparatus, comprising:

23. An electronic device, comprising:

a processor;

24. A storage device that stores a program of a method of obtaining a data reconstruction model, the program being executed by a processor and performing the steps of:

25. An abnormal data detection apparatus for a network platform, comprising:

26. An electronic device, comprising:

a processor;

27. A storage device storing a program of an abnormal data detection method for a network platform, the program being executed by a processor and performing the steps of:

28. An abnormality detection apparatus for a target device, characterized by comprising:

29. An electronic device, comprising:

a processor;

acquiring to-be-detected running state data of target equipment;

30. A storage device storing a program of an abnormality detection method for a target device, the program being executed by a processor and performing the steps of:

acquiring to-be-detected running state data of target equipment;