CN111835696A

CN111835696A - Method and device for detecting abnormal request individuals

Info

Publication number: CN111835696A
Application number: CN201910327904.8A
Authority: CN
Inventors: 韩啸; 赵征; 郭志强; 刘添龙
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-04-23
Filing date: 2019-04-23
Publication date: 2020-10-27
Anticipated expiration: 2039-04-23
Also published as: CN111835696B

Abstract

The application discloses a method and a device for detecting an abnormal request individual, wherein the method comprises the following steps: obtaining abnormal request data existing in the original request data; determining a request individual to which the abnormal request data belongs; acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals. By adopting the method for detecting the abnormal request individuals, the potential abnormal request individuals existing in the request individuals can be determined, the accuracy of abnormal detection is improved, and the data security of a business system is effectively guaranteed.

Description

Method and device for detecting abnormal request individuals

Technical Field

The application relates to the field of data security protection, in particular to a method and a device for detecting an abnormal request individual. In addition, the application also relates to a method and a device for acquiring the abnormal request characteristics and a system for detecting the abnormal request characteristics.

Background

With the rapid development of network technology, internet-based service systems are becoming more and more widespread. Currently, many services can be implemented on the internet depending on a service system, for example: online banking, online shopping, online games, etc. Along with the appearance of various service systems, a large number of network security holes are generated, and an abnormal request individual can acquire the network security holes in a full-scanning mode, so that the service system is attacked, and illegal benefits are obtained by acquiring personal account information of other people. Therefore, identifying potential abnormal request individuals and guaranteeing the data security of business systems become a research hotspot in the internet field nowadays.

In order to solve the above technical problems, a solution generally adopted in the prior art is to preset a WAF (web application Firewall) Firewall to perform abnormal request detection, and identify and intercept request data sent by an abnormal request individual, so as to implement data security of a service system, thereby avoiding malicious intrusion of the abnormal request individual into the service system to a certain extent, and improving the abnormal protection capability of the service system.

However, the WAF firewall technology generally relies on the attack data that has already occurred to trigger, and then identifies the abnormal request individual to perform effective interception. The technology has obvious passivity, detection data is easy to be incomplete, potential abnormal request individuals cannot be found, and therefore the abnormal protection requirement of the current business system cannot be met.

Disclosure of Invention

The application provides a method for detecting an abnormal request individual, which aims to solve the problem that the current network abnormal protection requirement cannot be met due to the fact that potential abnormal attack request data are difficult to find in the prior art. The application also provides a device for detecting the abnormal request individuals.

The method for detecting the abnormal request individuals, provided by the invention, comprises the following steps: obtaining abnormal request data existing in the original request data; determining a request individual to which the abnormal request data belongs; acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals.

Optionally, the determining, according to the abnormal probability information of the requesting individual, an abnormal requesting individual existing in the requesting individual includes: determining a target request individual existing in the request individual according to the abnormal probability information of the request individual; and determining abnormal request individuals existing in the target request individuals according to the white list information of the historical request individuals.

Optionally, the determining, according to the abnormal probability information of the requesting individual, a target requesting individual existing in the requesting individual includes: determining a target request individual existing in the request individual according to the abnormal probability value of the request individual, wherein the target request individual is the request individual of which the abnormal probability value reaches or exceeds a preset probability threshold; or determining a target request individual existing in the request individual according to the abnormal probability level of the request individual, wherein the target request individual is the request individual of which the abnormal probability level reaches or exceeds a preset probability level.

Optionally, the determining a requesting individual to which the abnormal request data belongs includes: and determining the request individual to which the abnormal request data belongs according to the identification field for identifying the request individual in the abnormal request data.

Optionally, the determining, according to the identification field that identifies the requesting individual in the abnormal request data, the requesting individual to which the abnormal request data belongs includes: and searching abnormal request data sent by the request individuals corresponding to the identification fields based on the corresponding relation between the identification fields and the abnormal request data by taking the identification fields as indexes.

Optionally, the abnormal probability information of the request data sent by the request individual is an abnormal probability value of the request data sent by the request individual; the obtaining, according to the abnormal probability information of the request data sent by the request individual, the abnormal probability information of the request individual specifically includes: and calculating the average value of the abnormal probability value of each request datum sent by the request individual, and taking the average value as the abnormal probability value of the request individual.

Optionally, the abnormal probability information of the request data sent by the request individual is an abnormal probability level of the request data sent by the request individual; the obtaining, according to the abnormal probability information of the request data sent by the request individual, the abnormal probability information of the request individual specifically includes: and according to the distribution proportion of the request data sent by the request individual in the preset abnormal probability grades, taking the abnormal probability grade with the highest distribution proportion of the request data as the abnormal probability grade of the request individual.

Optionally, the determining, according to the abnormal probability information of the request individual, an abnormal request individual existing in the request individual or determining whether the request individual is the abnormal request individual specifically includes: judging whether the abnormal probability value of the request individual reaches the abnormal probability threshold value of the abnormal request individual, if so, determining the request individual corresponding to the abnormal probability value of the request individual as the abnormal request individual; or judging whether the abnormal probability level of the request individual reaches the target probability level of the abnormal request individual, and if so, determining the request individual corresponding to the abnormal probability level of the request individual as the abnormal request individual.

Optionally, the obtaining of the abnormal request data existing in the original request data includes: obtaining abnormal probability information of the original request data; and determining abnormal request data existing in the original request data according to the abnormal probability information of the original request data.

Optionally, the determining, according to the abnormal probability information of the original request data, abnormal request data existing in the original request data specifically includes: judging whether the abnormal probability value of the original request data reaches the abnormal probability threshold of the abnormal request data, if so, determining the request data corresponding to the abnormal probability value of the original request data as the abnormal request data; and judging whether the abnormal probability level of the original request data reaches the abnormal probability level of the abnormal request data, if so, determining the request data corresponding to the abnormal probability level of the original request data as the abnormal request data.

Optionally, the identification field includes at least one of a device ID data field, a Cookie data field, an IP address data field, and a website account association data field for identifying the requesting individual.

Correspondingly, the application also provides a method for acquiring the abnormal request characteristics, which comprises the following steps: acquiring original request data sent by an abnormal request individual; obtaining abnormal request data sent by the abnormal request individuals from the original request data; and obtaining unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics.

Optionally, the obtaining unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics includes: obtaining abnormal request characteristics of the abnormal request data; and matching the abnormal request features of the abnormal request data with the known abnormal request features, and if the abnormal request features of the abnormal request data cannot be matched with the known abnormal request features, determining the abnormal request features of the abnormal request data as unknown abnormal request features.

Optionally, the obtaining unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics includes: obtaining known abnormal request data according to the known abnormal request characteristics; obtaining unknown exception request data other than the known exception request data from the exception request data; and acquiring the unknown abnormal request characteristics according to the unknown abnormal request data.

Optionally, the method for obtaining the abnormal request feature further includes: training an anomaly detection model by taking the unknown anomaly request characteristics as training samples; the abnormal detection model is used for judging whether the request data with the unknown abnormal request characteristics is abnormal request data according to the request characteristics.

Optionally, the obtaining, from the original request data, the abnormal request data sent by the abnormal request individual includes: and obtaining the abnormal request data sent by the abnormal request individuals from the original request data based on a white traffic baseline technology.

Correspondingly, the application provides a device for detecting an abnormal request individual, which comprises: the abnormal request data obtaining unit is used for obtaining abnormal request data existing in the original request data; a request individual determining unit, configured to determine a request individual to which the abnormal request data belongs; the abnormal probability information obtaining unit is used for obtaining the abnormal probability information of the request individuals according to the abnormal probability information of the request data sent by the request individuals; and the abnormal request individual determining unit is used for determining an abnormal request individual existing in the request individual or determining whether the request individual is the abnormal request individual according to the abnormal probability information of the request individual.

Optionally, the abnormal request individual determining unit includes: the target request individual determining subunit is used for determining a target request individual existing in the request individual according to the abnormal probability information of the request individual; and the abnormal request individual determining subunit is used for determining the abnormal request individual existing in the target request individual according to the white list information of the historical request individual.

Optionally, the target request individual determination subunit is specifically configured to: determining a target request individual existing in the request individual according to the abnormal probability value of the request individual, wherein the target request individual is the request individual of which the abnormal probability value reaches or exceeds a preset probability threshold; or determining a target request individual existing in the request individual according to the abnormal probability level of the request individual, wherein the target request individual is the request individual of which the abnormal probability level reaches or exceeds a preset probability level.

Optionally, the request individual determining unit is specifically configured to: and determining the request individual to which the abnormal request data belongs according to the identification field for identifying the request individual in the abnormal request data.

Optionally, the abnormal probability information of the request data sent by the request individual is an abnormal probability value of the request data sent by the request individual; the anomaly probability information obtaining unit is specifically configured to: and calculating the average value of the abnormal probability value of each request datum sent by the request individual, and taking the average value as the abnormal probability value of the request individual.

Optionally, the abnormal probability information of the request data sent by the request individual is an abnormal probability level of the request data sent by the request individual; the anomaly probability information obtaining unit is specifically configured to: and according to the distribution proportion of the request data sent by the request individual in the preset abnormal probability grades, taking the abnormal probability grade with the highest distribution proportion of the request data as the abnormal probability grade of the request individual.

Optionally, the abnormal request individual determining unit is specifically configured to: judging whether the abnormal probability value of the request individual reaches the abnormal probability threshold value of the abnormal request individual, if so, determining the request individual corresponding to the abnormal probability value of the request individual as the abnormal request individual; or judging whether the abnormal probability level of the request individual reaches the target probability level of the abnormal request individual, and if so, determining the request individual corresponding to the abnormal probability level of the request individual as the abnormal request individual.

Optionally, the different request data obtaining unit is specifically configured to: obtaining abnormal probability information of the original request data; and determining abnormal request data existing in the original request data according to the abnormal probability information of the original request data.

Correspondingly, the present application also provides a device for obtaining the abnormal request feature, including: the original request data obtaining unit is used for obtaining original request data sent by the abnormal request individuals; an abnormal request data obtaining unit, configured to obtain, from the original request data, abnormal request data sent by the abnormal request individual; and the unknown abnormal request characteristic obtaining unit is used for obtaining the unknown abnormal request characteristic from the abnormal request data according to the known abnormal request characteristic.

Optionally, the unknown abnormal request feature obtaining unit is specifically configured to obtain an abnormal request feature of the abnormal request data; and matching the abnormal request features of the abnormal request data with the known abnormal request features, and if the abnormal request features of the abnormal request data cannot be matched with the known abnormal request features, determining the abnormal request features of the abnormal request data as unknown abnormal request features.

Optionally, the unknown abnormal request feature obtaining unit is specifically configured to obtain the unknown abnormal request feature from the abnormal request data, and includes: obtaining known abnormal request data according to the known abnormal request characteristics; obtaining unknown exception request data other than the known exception request data from the exception request data; and acquiring the unknown abnormal request characteristics according to the unknown abnormal request data.

Optionally, the apparatus for obtaining an exception request feature further includes: the training unit is used for training an anomaly detection model by taking the unknown anomaly request characteristics as training samples; the abnormal detection model is used for judging whether the request data with the unknown abnormal request characteristics is abnormal request data according to the request characteristics.

Optionally, the abnormal request data obtaining unit is specifically configured to: and obtaining the abnormal request data sent by the abnormal request individuals from the original request data based on a white traffic baseline technology.

Correspondingly, the present application also provides a system for detecting an abnormal request feature, which is characterized by comprising: the device for detecting the abnormal request individual and the device for acquiring the abnormal request characteristic.

Accordingly, the present application provides an electronic device comprising: a processor and a memory; the memory is used for storing a program of a method for detecting an abnormal request individual, and after the equipment is powered on and runs the program of the method for detecting the abnormal request individual through the processor, the following steps are executed: obtaining abnormal request data existing in the original request data; determining a request individual to which the abnormal request data belongs; acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals.

Accordingly, the present application provides a storage device storing a program of a method of detecting an abnormally requested individual, the program being executed by a processor and performing the steps of: obtaining abnormal request data existing in the original request data; determining a request individual to which the abnormal request data belongs; acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals.

Compared with the prior art, the method has the following advantages:

by adopting the method for detecting the abnormal request individuals, the request individuals to which the abnormal request data belong can be determined by obtaining the abnormal request data existing in the original request data; acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining potential abnormal request individuals existing in the request individuals according to the abnormal probability information of the request individuals, and improving the accuracy of abnormal detection, thereby effectively ensuring the data security of a business system.

By adopting the method for acquiring the unknown abnormal request characteristics, the abnormal request data sent by the abnormal request individuals can be acquired from the original request data by acquiring the original request data sent by the abnormal request individuals, and the potential unknown abnormal request characteristics can be acquired from the abnormal request data according to the known abnormal request characteristics, so that the integrity and the accuracy of abnormal detection are improved, and the data security of a service system is effectively guaranteed.

Drawings

FIG. 1 is a flowchart of a method for detecting an abnormal requesting individual according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating an apparatus for detecting an abnormal requesting individual according to an embodiment of the present invention;

fig. 3 is a flowchart of a method for obtaining unknown abnormal request features according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a method for obtaining unknown abnormal request features according to an embodiment of the present invention;

fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present invention;

fig. 6 is a flowchart illustrating a complete operation of a network security protection system according to an embodiment of the present invention;

fig. 7 is a schematic application scenario diagram of a method for detecting an abnormal request individual according to an embodiment of the present invention.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather construed as limited to the embodiments set forth herein.

The following describes embodiments of the method for detecting an abnormal request individual according to the present invention in detail. Fig. 7 is a schematic view of an application scenario of a method for detecting an abnormal request individual according to an embodiment of the present invention.

In a specific implementation process, the method for detecting an abnormal request individual according to the present invention can be implemented based on the conventional mobile terminal 701 carrying a network data security protection system. For example, the mobile terminal 701 obtains the abnormal request data existing in the original request data, determines the request individual to which the abnormal request data belongs, and further obtains the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining the potential abnormal request individuals existing in the request individuals through the abnormal probability information of the request individuals.

In addition, the method for detecting an abnormal request individual according to the embodiment of the present invention can also be performed based on the mobile terminal 701 and the server 702 with the network data security system. For example, the mobile terminal 701 obtains a large amount of network traffic data, and sends the network traffic data to the server 702 carrying the network data security protection system. The server 702 may obtain original request data according to network traffic data analysis, further obtain abnormal request data existing in the original request data, and determine a request individual to which the abnormal request data belongs; further acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals.

It should be noted that the method for detecting an abnormal request individual according to the present invention can be applied to protect a Web server, and in general, is applied to a front end of a Web server that provides a Web service to the outside.

Fig. 1 is a flowchart illustrating a method for detecting an abnormal requesting individual according to an embodiment of the present invention. The method for detecting the abnormal request individuals comprises the following steps:

step S101: and obtaining abnormal request data existing in the original request data.

In the embodiment of the present invention, the original request data may be service traffic data sent based on a hypertext Transfer Protocol (HTTP) and accessed to a website or an APP application. The abnormal request data may refer to request data that is obtained from original request data by using a specific extraction method and has a threat or attack behavior on a website or an APP application, for example: XSS (Cross Site Scripting) attack, SQL (Structured Query Language) injection, CSRF (Cross-Site request forgery) attack, Cookie attack, Xpath (XML Path Language) injection, LDAP (Lightweight Directory Access Protocol) injection, code execution, and the like. The specific extraction method can be used for classifying the total original request data of the user based on a white traffic baseline technology, separating normal request data which have no attack behaviors and occupy a large probability, and taking the residual original request data with a small probability as abnormal request data. In the embodiment of the present invention, the abnormal request data may be request data in which the abnormal probability information reaches or exceeds a preset abnormal probability information threshold.

Specifically, the obtaining of the abnormal request data existing in the original request data obtains the abnormal probability information of the original request data, and then determines the abnormal request data existing in the original request data according to the abnormal probability information of the original request data. The method comprises the following steps of determining abnormal request data existing in original request data according to abnormal probability information of the original request data, and realizing the following specific modes:

and judging whether the abnormal probability value of the original request data reaches the abnormal probability threshold of the abnormal request data, if so, determining the request data corresponding to the abnormal probability value of the original request data as the abnormal request data. Or judging whether the abnormal probability level of the original request data reaches the abnormal probability level of the abnormal request data, and if so, determining the request data corresponding to the abnormal probability level of the original request data as the abnormal request data.

It should be noted that, in the actual application process, various request data for the website or APP application may be collected based on the front-end client, sometimes a data log set corresponding to the request data may be directly collected, and the original request data capable of representing the original meaning of the request data is obtained based on the log data in the data log set. Sometimes, the data traffic packets are collected, however, the original meaning of the request data cannot be accurately identified directly according to the data traffic packets, and further, the abnormal request data existing therein cannot be detected, so that when the data traffic packets are collected, the data traffic packets need to be further subjected to code conversion, and the network traffic data is converted into a combination of metadata, that is, the metadata is obtained; and carrying out protocol decoding on the network traffic data on the protocol layer to obtain the original request data which can represent the original meaning of the request data.

Step S102: and determining the request individual to which the abnormal request data belongs.

After the abnormal request data existing in the original request data is obtained in step S101, data preparation work is performed for determining the request individual to which the abnormal request data belongs in this step. In step S102, the requesting entity to which the abnormal request data belongs may be determined according to the identification field of the corresponding requesting entity contained in the abnormal request data.

Specifically, the determination of the requesting individual to which the abnormal request data belongs according to the identification field that identifies the requesting individual in the abnormal request data can be implemented based on the following specific manner: and searching abnormal request data sent by the request individuals corresponding to the identification fields from the abnormal request data by taking the identification fields corresponding to the request individuals contained in the abnormal request data as indexes and based on the corresponding relation between the identification fields and the abnormal request data. The identification field comprises at least one of a device ID data field, a Cookie data field, an IP address data field and a website account association data field for identifying the request individual.

Step S103: and acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual.

After the step S102 determines the request individual to which the abnormal request data belongs according to the identification field of the corresponding request individual contained in the abnormal request data, the step may obtain the abnormal probability information of the request individual based on the abnormal probability information of the request data sent by the request individual.

In the embodiment of the present invention, the request data sent by the requesting individual may be scored through an HMM algorithm or an SVM algorithm, that is: and acquiring the abnormal probability information of the request data sent by the request individual. Of course, other anomaly detection algorithms may be used to obtain the anomaly probability information of the request data many times, and are not described in detail here.

After determining the request individual to which the abnormal request data belongs and the abnormal probability information of the request data sent by the request individual, the request data can be summarized and counted based on the device ID data field, the Cookie data field, the IP address data field and the like of the request individual, so as to obtain the abnormal probability information of the request individual. IP fingerprints, website account association information, etc. are also used many times.

Specifically, the abnormal probability information of the request data sent by the requesting individual may refer to an abnormal probability value of the request data sent by the requesting individual, or may refer to an abnormal probability level of the request data sent by the requesting individual.

When the abnormal probability information of the request data sent by the requesting individual may refer to the abnormal probability value of the request data sent by the requesting individual, the average value of the abnormal probability values of each request data sent by the requesting individual may be calculated, and the average value is used as the abnormal probability value of the requesting individual. For example; the abnormal probability values of 7 pieces of request data sent by the individual A request are respectively: 0.8, 0.9, 0.7, 0.8, 0.6, 0.1, 0.9, and calculating the average value of the abnormal probability values of each request data sent by the requesting individuals as follows: 0.68, i.e., the abnormal probability value of the requesting individual is 0.68. The abnormal probability values of the 9 pieces of request data sent by the individual B request are respectively: 0.8, 0.7, 0.9, 0.6, 0.8, 0.9, 0.7, and calculating an average value of the abnormality probability values of each request data issued by the requesting individual as: 0.78, i.e., the anomaly probability value for the requesting individual is 0.68. The abnormal probability values of 6 pieces of request data sent by the C request individual are respectively: 0.3, 0.1, 0.6, 0.1, 0.2, 0.3, and calculating the average value of the abnormal probability values of each request data sent by the request individuals as follows: 0.68, i.e., the abnormal probability value of the requesting individual is 0.26.

When the abnormal probability information of the request data sent by the requesting individual may refer to an abnormal probability level of the request data sent by the requesting individual, the abnormal probability level with the highest request data distribution ratio may be used as the abnormal probability level of the requesting individual according to a distribution ratio of the request data sent by the requesting individual in a preset abnormal probability level. For example; the preset abnormal probability grades are divided into: high, medium, low; if the request data sent by the individual requesting D is 100, wherein 74 pieces of request data are determined to have high abnormal probability level, 16 pieces of request data are determined to have medium abnormal probability level, and 10 pieces of request data are determined to have low abnormal probability level, the abnormal probability level of the individual requesting D can be determined to be high according to the distribution proportion condition of the request data. The request data sent by the individual E-request is 70, wherein 7 pieces of request data are determined to have high anomaly probability levels, 62 pieces of request data are determined to have medium anomaly probability levels, and 1 piece of request data is determined to have low anomaly probability levels, and the anomaly probability level of the individual E-request can be determined to be medium according to the distribution proportion of the request data. If the request data sent by the individual request F is 200, wherein 16 pieces of request data are determined to have high anomaly probability levels, 10 pieces of request data are determined to have medium anomaly probability levels, and 174 pieces of request data are determined to have medium anomaly probability levels, the anomaly probability level of the individual request F can be determined to be low according to the distribution proportion condition of the request data.

Step S104: and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals.

After the step S103 obtains the abnormal probability information of the requesting individual according to the abnormal probability information of the request data sent by the requesting individual, the step may further determine the abnormal requesting individual existing in the requesting individual based on the abnormal probability information of the requesting individual.

Specifically, determining the abnormal requesting individual existing in the requesting individual according to the abnormal probability information of the requesting individual can be implemented by the following steps:

1) and determining whether the abnormal request individuals exist in the request individuals or not according to the size of the abnormal probability value or the level of the abnormal probability of the request individuals.

Specifically, when the abnormal probability information of the request data is the abnormal probability value of the request data, whether the abnormal probability value of the request individual reaches the abnormal probability threshold of the abnormal request individual is judged, and if yes, the request individual corresponding to the abnormal probability value of the request individual is determined to be the abnormal request individual. For example: obtaining the abnormal probability value of the individual A request to be 0.68 through calculation; the abnormal probability value of the individual B request is 0.78; c the anomaly probability value for the requesting individual is 0.26. And if the abnormal probability threshold of the abnormal request individual is 0.6, determining that the request individual A and the request individual B corresponding to the abnormal probability values of 0.68 and 0.78 of the request individual are abnormal request individuals.

And when the abnormal probability information of the request data is the abnormal probability level of the request data, judging whether the abnormal probability level of the request individual reaches the target probability level of the abnormal request individual, and if so, determining the request individual corresponding to the abnormal probability level of the request individual as the abnormal request individual. For example: obtaining the abnormal probability grade of the individual D request through analysis; e, requesting the abnormal probability grade of the individual to be a middle grade; f, requesting the abnormal probability level of the individual to be low; and if the target probability level of the abnormal request individual is the middle level, determining the request individual D with the high abnormal probability level of the request individual and the request individual E with the middle abnormal probability level of the request individual as the abnormal request individual.

2) Determining a target request individual existing in the request individual according to the abnormal probability information of the request individual; and determining abnormal request individuals existing in the target request individuals according to the white list information of the historical request individuals.

Similarly, the abnormal probability information of the request data sent by the requesting individual may refer to an abnormal probability value of the request data sent by the requesting individual, or may refer to an abnormal probability level of the request data sent by the requesting individual. The target request individual may be a request individual whose abnormal probability level reaches a preset probability level or whose abnormal probability value reaches a preset probability value. For example, the target requesting individual may refer to the requesting individual a and the requesting individual B corresponding to the above abnormal probability values 0.68 and 0.78, respectively, or may refer to the requesting individual D having a high abnormal probability level and the requesting individual E having a medium abnormal probability level. If the target request individual is the request individual A and the request individual B, the target request individual can be further filtered by using the white list information of the historical request individual to obtain the abnormal request individual existing in the target request individual. The history request individual white list information may refer to known characteristic information of a normal request individual. For example: and based on the historical request individual white list information, the request individual B can be determined to be a normal request individual, and the finally remaining request individuals A are detected abnormal request individuals. It should be noted that, in the actual implementation process, the request individual a or the request individual B is not limited to refer to a specific request individual, and may refer to multiple request individuals; the request data can be the full request data set of the requesting individual.

Corresponding to the method for detecting the abnormal request individuals, the invention also provides a device for detecting the abnormal request individuals. Since the embodiment of the apparatus is similar to the above method embodiment, the description is simple, and for the relevant points, reference may be made to the description of the above method embodiment, and the following description of an apparatus for detecting an abnormal request individual is only illustrative. Fig. 2 is a schematic diagram of an apparatus for detecting an abnormal request individual according to an embodiment of the present invention.

The device for detecting the abnormal request individuals comprises the following parts:

an abnormal request data obtaining unit 201, configured to obtain abnormal request data existing in the original request data.

A requesting individual determining unit 202, configured to determine a requesting individual to which the abnormal request data belongs.

After the above-mentioned abnormal request data obtaining unit 201 obtains the abnormal request data existing in the original request data, it performs data preparation work for the requesting entity determining unit 202 to determine the requesting entity to which the abnormal request data belongs. In the requesting entity determining unit 202, the requesting entity to which the abnormal request data belongs may be determined according to the identification field of the corresponding requesting entity contained in the abnormal request data.

The abnormal probability information obtaining unit 203 is configured to obtain abnormal probability information of the requesting individual according to the abnormal probability information of the request data sent by the requesting individual.

After the request individual determining unit 202 determines the request individual to which the abnormal request data belongs, the abnormal probability information obtaining unit 203 obtains the abnormal probability information of the request individual by analyzing the abnormal probability information of the request data sent by the request individual.

An abnormal request individual determining unit 204, configured to determine an abnormal request individual existing in the request individual or determine whether the request individual is the abnormal request individual according to the abnormal probability information of the request individual.

After the request individual determining unit 203 obtains the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual, further, the abnormal request individual determining unit 204 can determine the abnormal request individual existing in the request individual according to the abnormal probability information of the request individual.

By adopting the device for detecting the abnormal request individuals, the abnormal request data existing in the original request data can be obtained through the abnormal request data obtaining unit, and the request individuals to which the abnormal request data belong are determined through the request individual determining unit; obtaining the abnormal probability information of the request individual by an abnormal probability information obtaining unit according to the abnormal probability information of the request data sent by the request individual; the abnormal request individual determining unit determines potential abnormal request individuals existing in the request individuals according to the abnormal probability information of the request individuals, and improves the accuracy of abnormal detection, so that the data safety of a business system is effectively guaranteed.

Corresponding to the method for detecting the abnormal request individual, the invention also provides electronic equipment. Fig. 3 is a schematic view of an electronic device according to an embodiment of the invention.

An electronic device provided by an embodiment of the application includes the following parts: a processor 501 and a memory 502, wherein the memory 502 is used for storing a program 503 of a method for detecting an abnormal request individual, and after the device is powered on and the program 503 of the method for detecting the abnormal request individual is run by the processor 501, the following steps are executed: obtaining abnormal request data existing in the original request data; determining a request individual to which the abnormal request data belongs; acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals. The processor 501 and the memory 502 perform data transmission based on a bus 505 established therebetween, and the communication interface 504 is a data exchange interface for connecting the electronic device with the outside. It should be noted that, for the detailed description of the electronic device provided in the embodiment of the present application, reference may be made to the related description of the method for detecting an abnormal request individual provided in the embodiment of the present application, and details are not repeated here.

Corresponding to the method for detecting the abnormal request individual, the invention also provides a storage device, wherein the storage device stores a program of the method for detecting the abnormal request individual, the program is run by a processor and executes the following steps: obtaining abnormal request data existing in the original request data; step two: determining a request individual to which the abnormal request data belongs; step three: acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual; step four: and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals. It should be noted that, for the detailed description of a storage device provided in the embodiment of the present application, reference may be made to the related description of a method for detecting an abnormal request individual provided in the embodiment of the present application, and details are not repeated here.

Corresponding to the method for detecting the abnormal request individuals, the invention also provides a method for acquiring the abnormal request characteristics. The following describes an embodiment of the method for obtaining the abnormal request feature according to the present invention in detail. In a specific implementation process, the method for detecting the abnormal request individual, which is implemented by the application, is similar to the specific implementation process of the method for detecting the abnormal request individual, and can be implemented based on a traditional mobile terminal loaded with a network data security protection system; the method can also be carried out on the basis of the mobile terminal and a server carrying a network data security protection system.

Please refer to fig. 3, which is a flowchart illustrating a method for obtaining unknown abnormal request features according to an embodiment of the present invention.

Step S301: and obtaining original request data sent by the abnormal request individuals.

In the embodiment of the present invention, the abnormal request individual is a request individual to which the abnormal request data belongs, which is determined by obtaining the abnormal request data existing in the original request data, the abnormal probability information of the request individual is obtained according to the abnormal probability information of the request data sent by the request individual, and the abnormal request individual existing in the request individual is determined according to the abnormal probability information of the request individual. The original request data sent by the abnormal request individuals can be obtained by carrying out statistical analysis by using the identification fields which are used for identifying the abnormal request individuals and exist in the original request data.

Step S302: and obtaining abnormal request data sent by the abnormal request individuals from the original request data.

In the embodiment of the invention, the abnormal request data sent by the abnormal request individual can be obtained from the original request data based on a white traffic baseline technology. It should be noted that, in many cases, other similar technical means may also be adopted to separate the abnormal request data from the original request data, and details are not repeated here.

Step S303: and obtaining unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics.

In the embodiment of the present invention, obtaining the unknown abnormal request feature from the abnormal request data according to the known abnormal request feature can be implemented by the following two ways:

1) and obtaining the abnormal request features of the abnormal request data, matching the abnormal request features of the abnormal request data with the known abnormal request features, and if the abnormal request features of the abnormal request data cannot be matched with the known abnormal request features, determining the abnormal request features of the abnormal request data as unknown abnormal request features.

2) Obtaining known abnormal request data according to the known abnormal request characteristics; obtaining unknown abnormal request data except the known abnormal request data from the abnormal request data; and acquiring the unknown abnormal request characteristics according to the unknown abnormal request data.

Further, the unknown anomaly request features can be used as training samples to train an anomaly detection model. The abnormal detection model is used for judging whether the request data with unknown abnormal request characteristics is abnormal request data according to the request characteristics. It should be noted that the anomaly detection model according to the present invention is not limited to the deep neural network model, and may also be implemented by a logistic regression model. In a specific implementation, a suitable model may be selected by a performance comparison test.

Fig. 6 is a flowchart illustrating a complete operation of a network security protection system according to an embodiment of the present invention. The complete operation process based on the embodiment of the invention can comprise the following steps: firstly, network flow data and preprocessing are carried out; network flow data is collected at a client by means of acquisition of network flow data capture, existing data collection and the like, and an original request data set is obtained based on the collected network flow data and a log set of the network flow data. Secondly, requesting data scoring; and scoring the original request data in the original request data set based on a white traffic baseline technology to obtain abnormal request data. Thirdly, detecting an abnormal request individual; and acquiring abnormal probability information of the request individuals based on the abnormal probability information of the abnormal request data sent by the request individuals, and further acquiring abnormal request individuals existing in the request individuals according to the abnormal probability information of the request individuals. Fourthly, acquiring unknown abnormal request characteristics; and taking the identification information of the abnormal request individuals as an index, backtracking the original request data set to obtain the total request data sent by the abnormal request individuals, filtering the confused abnormal request data by utilizing a white flow baseline technology, and further filtering the abnormal request data according to the known abnormal request characteristics to obtain unknown abnormal request characteristics. The fifth step: training an anomaly detection model; and training an anomaly detection model by taking the unknown anomaly request characteristics as training samples.

Fig. 4 is a schematic diagram illustrating a method for obtaining unknown abnormal request features according to an embodiment of the present invention.

Original request data obtaining unit 401: and obtaining original request data sent by the abnormal request individuals.

The exception request data obtaining unit 402: and obtaining abnormal request data sent by the abnormal request individuals from the original request data.

Unknown anomaly request feature obtaining unit 403: and obtaining unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics.

By adopting the device for acquiring the unknown abnormal request characteristics, the original request data sent by the abnormal request individuals can be acquired through the original request data acquisition unit, the abnormal request data acquisition unit acquires the abnormal request data sent by the abnormal request individuals from the original request data, and the unknown abnormal request characteristics acquisition unit acquires the potential unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics, so that the integrity and the accuracy of abnormal detection are improved, and the data safety of a service system is effectively guaranteed.

Although the present invention has been described with reference to the preferred embodiments, it is not intended to limit the present invention, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present invention.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims

1. A method for detecting an abnormally requesting individual, comprising:

obtaining abnormal request data existing in the original request data;

determining a request individual to which the abnormal request data belongs;

acquiring the abnormal probability information of the request individual according to the abnormal probability information of the request data sent by the request individual;

and determining abnormal request individuals existing in the request individuals or determining whether the request individuals are the abnormal request individuals according to the abnormal probability information of the request individuals.

2. The method for detecting an abnormal request individual according to claim 1, wherein the determining the abnormal request individual existing in the request individual according to the abnormal probability information of the request individual comprises:

determining a target request individual existing in the request individual according to the abnormal probability information of the request individual;

and determining abnormal request individuals existing in the target request individuals according to the white list information of the historical request individuals.

3. The method for detecting the abnormal request individuals according to claim 2, wherein the determining the target request individuals existing in the request individuals according to the abnormal probability information of the request individuals comprises:

determining a target request individual existing in the request individual according to the abnormal probability value of the request individual, wherein the target request individual is the request individual of which the abnormal probability value reaches or exceeds a preset probability threshold;

or determining a target request individual existing in the request individual according to the abnormal probability level of the request individual, wherein the target request individual is the request individual of which the abnormal probability level reaches or exceeds a preset probability level.

4. The method for detecting an abnormal request individual according to claim 1, wherein the determining the request individual to which the abnormal request data belongs comprises:

and determining the request individual to which the abnormal request data belongs according to the identification field for identifying the request individual in the abnormal request data.

5. The method for detecting the abnormal request individuals according to claim 4, wherein the determining the request individuals to which the abnormal request data belongs according to the identification fields for identifying the request individuals in the abnormal request data comprises:

and searching abnormal request data sent by the request individuals corresponding to the identification fields based on the corresponding relation between the identification fields and the abnormal request data by taking the identification fields as indexes.

6. The method for detecting the abnormal intrusion individual according to claim 1, wherein the abnormal probability information of the request data sent by the requesting individual is an abnormal probability value of the request data sent by the requesting individual;

the obtaining, according to the abnormal probability information of the request data sent by the request individual, the abnormal probability information of the request individual specifically includes:

and calculating the average value of the abnormal probability value of each request datum sent by the request individual, and taking the average value as the abnormal probability value of the request individual.

7. The method for detecting the abnormal intrusion individual according to claim 1, wherein the abnormal probability information of the request data sent by the requesting individual is an abnormal probability level of the request data sent by the requesting individual;

and according to the distribution proportion of the request data sent by the request individual in the preset abnormal probability grades, taking the abnormal probability grade with the highest distribution proportion of the request data as the abnormal probability grade of the request individual.

8. The method for detecting an abnormal invading individual according to claim 1, wherein said determining abnormal requesting individual existing in said requesting individual according to the abnormal probability information of said requesting individual specifically comprises:

judging whether the abnormal probability value of the request individual reaches the abnormal probability threshold value of the abnormal request individual, if so, determining the request individual corresponding to the abnormal probability value of the request individual as the abnormal request individual;

or judging whether the abnormal probability level of the request individual reaches the target probability level of the abnormal request individual, and if so, determining the request individual corresponding to the abnormal probability level of the request individual as the abnormal request individual.

9. The method for detecting abnormal request individuals according to claim 1, wherein the obtaining abnormal request data existing in original request data comprises:

obtaining abnormal probability information of the original request data;

and determining abnormal request data existing in the original request data according to the abnormal probability information of the original request data.

10. The method for detecting an abnormal request individual according to claim 9, wherein the determining abnormal request data existing in the original request data according to the abnormal probability information of the original request data specifically comprises:

judging whether the abnormal probability value of the original request data reaches the abnormal probability threshold of the abnormal request data, if so, determining the request data corresponding to the abnormal probability value of the original request data as the abnormal request data;

and judging whether the abnormal probability level of the original request data reaches the abnormal probability level of the abnormal request data, if so, determining the request data corresponding to the abnormal probability level of the original request data as the abnormal request data.

11. The method for detecting the abnormal request individual according to the claim 4 or 5, wherein the identification field comprises at least one of a device ID data field, a Cookie data field, an IP address data field and a website account association data field for identifying the request individual.

12. A method for obtaining an exception request feature, comprising:

acquiring original request data sent by an abnormal request individual;

obtaining abnormal request data sent by the abnormal request individuals from the original request data;

and obtaining unknown abnormal request characteristics from the abnormal request data according to the known abnormal request characteristics.

13. The method for obtaining the abnormal request feature of claim 12, wherein the obtaining the unknown abnormal request feature from the abnormal request data according to the known abnormal request feature comprises:

obtaining abnormal request characteristics of the abnormal request data;

and matching the abnormal request features of the abnormal request data with the known abnormal request features, and if the abnormal request features of the abnormal request data cannot be matched with the known abnormal request features, determining the abnormal request features of the abnormal request data as unknown abnormal request features.

14. The method for obtaining the abnormal request feature of claim 12, wherein the obtaining the unknown abnormal request feature from the abnormal request data according to the known abnormal request feature comprises:

obtaining known abnormal request data according to the known abnormal request characteristics; obtaining unknown exception request data other than the known exception request data from the exception request data;

and acquiring the unknown abnormal request characteristics according to the unknown abnormal request data.

15. The method for obtaining an exception request feature of claim 12, further comprising:

training an anomaly detection model by taking the unknown anomaly request characteristics as training samples;

the abnormal detection model is used for judging whether the request data with the unknown abnormal request characteristics is abnormal request data according to the request characteristics.

16. The method for obtaining the abnormal request feature of claim 12, wherein the obtaining the abnormal request data issued by the abnormal request individual from the original request data comprises:

and obtaining the abnormal request data sent by the abnormal request individuals from the original request data based on a white traffic baseline technology.

17. An apparatus for detecting an abnormally requesting individual, comprising:

the abnormal request data obtaining unit is used for obtaining abnormal request data existing in the original request data;

a request individual determining unit, configured to determine a request individual to which the abnormal request data belongs;

the abnormal probability information obtaining unit is used for obtaining the abnormal probability information of the request individuals according to the abnormal probability information of the request data sent by the request individuals;

and the abnormal request individual determining unit is used for determining an abnormal request individual existing in the request individual or determining whether the request individual is the abnormal request individual according to the abnormal probability information of the request individual.

18. An apparatus for obtaining exception request features, comprising:

the original request data obtaining unit is used for obtaining original request data sent by the abnormal request individuals;

an abnormal request data obtaining unit, configured to obtain, from the original request data, abnormal request data sent by the abnormal request individual;

and the unknown abnormal request characteristic obtaining unit is used for obtaining the unknown abnormal request characteristic from the abnormal request data according to the known abnormal request characteristic.

19. A system for detecting anomalous request features, comprising: the apparatus for detecting an abnormal request individual according to claim 13, and the apparatus for acquiring the characteristics of the abnormal request according to claim 14.

20. An electronic device, comprising:

a processor; and

a memory for storing a program of a method of detecting an abnormality requesting individual, the apparatus being powered on and executing the program of the method of detecting an abnormality requesting individual by the processor, and performing the steps of:

obtaining abnormal request data existing in the original request data;

determining a request individual to which the abnormal request data belongs;

21. A storage device storing a program for a method of detecting an abnormally requested individual, the program being executed by a processor and performing the steps of:

obtaining abnormal request data existing in the original request data;

determining a request individual to which the abnormal request data belongs;