CN117131445A - Abnormal transaction detection method and system - Google Patents
Abnormal transaction detection method and system Download PDFInfo
- Publication number
- CN117131445A CN117131445A CN202310953489.3A CN202310953489A CN117131445A CN 117131445 A CN117131445 A CN 117131445A CN 202310953489 A CN202310953489 A CN 202310953489A CN 117131445 A CN117131445 A CN 117131445A
- Authority
- CN
- China
- Prior art keywords
- transaction
- abnormal
- data
- confidence interval
- knowledge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 246
- 238000001514 detection method Methods 0.000 title claims description 37
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000004458 analytical method Methods 0.000 claims abstract description 28
- 238000005065 mining Methods 0.000 claims abstract description 27
- 230000006399 behavior Effects 0.000 claims description 129
- 238000012549 training Methods 0.000 claims description 23
- 238000010276 construction Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 description 29
- 238000009826 distribution Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 13
- 238000003860 storage Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 11
- 238000013473 artificial intelligence Methods 0.000 description 9
- 230000001976 improved effect Effects 0.000 description 7
- 238000010801 machine learning Methods 0.000 description 7
- 238000004140 cleaning Methods 0.000 description 6
- 238000003058 natural language processing Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000000630 rising effect Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007418 data mining Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 206010000117 Abnormal behaviour Diseases 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 238000009960 carding Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010225 co-occurrence analysis Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000004900 laundering Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003012 network analysis Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012502 risk assessment Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Business, Economics & Management (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Animal Behavior & Ethology (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and a system for detecting abnormal transaction, comprising the steps of carrying out outlier mining analysis on acquired transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data; extracting features of normal transaction data in the transaction behavior confidence interval to obtain transaction behavior features in the transaction behavior confidence interval; and determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data, and further analyzing the abnormal transaction data to extract abnormal transaction behavior characteristics. The invention can analyze the transaction data and accurately identify abnormal transactions.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to an abnormal transaction detection method and system.
Background
With the rapid development of finance and information technology, security information security risks and business compliance risks are increasingly prominent, normal investment activities of clients and normal operation activities of securities companies are affected, compliance risks of operation institutions and industry supervision cost are increased, and adverse effects are brought to the healthy development of the securities market. At present, abnormal transaction identification is mainly carried out by adopting methods of manually screening a black-and-white list, configuring abnormal decision rules and the like. The black-white list and the decision rule depend on artificial experience, and artificial subjective factor interference exists, so that the recognition accuracy of abnormal transactions is difficult to improve.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the invention provides an abnormal transaction detection method and system, which can analyze transaction data and accurately identify abnormal transactions.
According to an aspect of an embodiment of the present invention, there is provided an abnormal transaction detection method including:
performing outlier mining analysis on the obtained transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data;
extracting features of normal transaction data in a transaction behavior confidence interval to obtain transaction behavior features in the transaction behavior confidence interval;
and determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data.
As an alternative embodiment, the determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data includes:
extracting characteristics of abnormal transaction data deviating from the transaction behavior confidence interval;
identifying the abnormal type of the abnormal transaction feature to obtain the abnormal type corresponding to the abnormal transaction feature; wherein the anomaly types comprise transaction behavior anomaly types and transaction error anomaly types;
If the abnormal type of the abnormal transaction characteristic is the abnormal type of the transaction behavior, determining the abnormal transaction characteristic as a core abnormal transaction characteristic;
and determining the transaction data corresponding to the core abnormal transaction characteristics as target abnormal transaction data.
As an alternative embodiment, after the determining the abnormal transaction characteristic as the core abnormal transaction characteristic, the method further includes:
carrying out full-quantity analysis on the transaction behavior characteristics in the transaction behavior confidence interval, and constructing a structured data set to obtain a first knowledge graph;
performing differential training on the core abnormal transaction characteristics and the transaction behavior characteristics in the transaction behavior confidence interval to obtain a second knowledge graph;
determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs;
model training is continuously carried out through a large amount of user transaction data, and knowledge graph construction is perfected.
As an optional implementation manner, after the determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs, the method further includes:
determining user information in the multi-type knowledge graph;
Extracting an associated knowledge graph of each piece of user information from the multi-type knowledge graph;
and obtaining the historical transaction information network relation of each user according to the associated knowledge graph.
As an optional implementation manner, after the obtaining the historical transaction information network relationship of each user, the method further includes:
deducing the historical transaction behaviors of each user, mining and identifying abnormal situations in the user transaction behaviors, and predicting abnormal events and occurrence probability of the predicted abnormal events according to the existing data of the user;
if the occurrence probability is larger than a preset probability threshold, generating and outputting abnormal event early warning information based on the predicted abnormal event corresponding to the occurrence probability.
According to another aspect of the embodiment of the present invention, there is also provided an abnormal transaction detection system including:
the analysis unit is used for carrying out outlier mining analysis on the acquired transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data;
the extraction unit is used for extracting the characteristics of the normal transaction data in the transaction behavior confidence interval to obtain the transaction behavior characteristics in the transaction behavior confidence interval;
And the determining unit is used for determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data.
As an alternative embodiment, the determining unit may determine the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data specifically includes:
extracting characteristics of abnormal transaction data deviating from the transaction behavior confidence interval;
identifying the abnormal type of the abnormal transaction feature to obtain the abnormal type corresponding to the abnormal transaction feature; wherein the anomaly types comprise transaction behavior anomaly types and transaction error anomaly types;
if the abnormal type of the abnormal transaction characteristic is the abnormal type of the transaction behavior, determining the abnormal transaction characteristic as a core abnormal transaction characteristic;
and determining the transaction data corresponding to the core abnormal transaction characteristics as target abnormal transaction data.
As an alternative embodiment, the determining unit is further configured to:
after the abnormal transaction characteristics are determined to be core abnormal transaction characteristics, carrying out full-quantity analysis on the transaction behavior characteristics in the transaction behavior confidence interval, and constructing a structured data set to obtain a first knowledge graph;
Performing differential training on the core abnormal transaction characteristics and the transaction behavior characteristics in the transaction behavior confidence interval to obtain a second knowledge graph;
determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs;
model training is continuously carried out through a large amount of user transaction data, and knowledge graph construction is perfected.
As an alternative embodiment, the determining unit is further configured to:
after the first knowledge-graph and the second knowledge-graph are determined to be multi-type knowledge-graphs, determining user information in the multi-type knowledge-graphs;
extracting an associated knowledge graph of each piece of user information from the multi-type knowledge graph;
and obtaining the historical transaction information network relation of each user according to the associated knowledge graph.
As an alternative embodiment, the determining unit is further configured to:
after obtaining the historical transaction information network relation of each user, deducing the historical transaction behavior of each user, mining and identifying abnormal situations in the transaction behavior of the user, and predicting abnormal events and occurrence probability of the predicted abnormal events according to the existing data of the user;
If the occurrence probability is larger than a preset probability threshold, generating and outputting abnormal event early warning information based on the predicted abnormal event corresponding to the occurrence probability.
According to yet another aspect of an embodiment of the present invention, there is also provided a computing device including: at least one processor, memory, and input output unit; the memory is used for storing a computer program, and the processor is used for calling the computer program stored in the memory to execute the abnormal transaction detection method.
According to yet another aspect of embodiments of the present invention, there is also provided a computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the above-described abnormal transaction detection method.
In the embodiment of the invention, outlier mining analysis can be performed on the acquired transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data; extracting features of normal transaction data in the transaction behavior confidence interval to obtain transaction behavior features in the transaction behavior confidence interval; determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data, and further analyzing and extracting abnormal transaction behavior characteristics through the abnormal transaction data; the transaction data can be analyzed, and abnormal transactions can be accurately identified.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of an alternative abnormal transaction detection method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an alternative abnormal transaction detection system according to an embodiment of the present application;
FIG. 3 schematically illustrates a structural diagram of abnormal transaction detection according to an embodiment of the present application;
FIG. 4 schematically illustrates an operational effect diagram of an abnormal transaction detection model according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiment of the invention can be applied to an abnormal transaction detection system, which mainly comprises: the data layer (a structured real-time digital bin is formed based on data cleaning and analysis of logs, businesses and the like), the algorithm layer (big data, machine learning, knowledge graph, natural language processing) and the application layer (anti-plug-in, user portraits and anomaly detection).
The data is a foundation, and an enormous data warehouse is constructed by gathering historical log information and business data information according to industrial data support; the algorithm is a core, a data warehouse is analyzed by utilizing big data and an artificial intelligence advanced technology, a structured storage data set is obtained, and a rich index library and a model library are established; the application is a valuable embodiment, and the model is converted into a product and applied to intelligent wind control.
High-efficiency big data technology is introduced, and the method is applied to data cleaning, data storage, data modeling and data application of huge data warehouse. By utilizing a data mining technology, a big data abnormal transaction analysis model is established, data samples required by intelligent abnormal transactions are analyzed, data cleaning, storage, modeling and application are performed in real time, service data and log data are applied in real time, the problem of information asymmetry is solved, and a high-quality, diversified and multi-dimensional sample model is provided for the system.
The four major techniques of artificial intelligence application core include feature recognition, machine learning, natural language processing, knowledge graph. The feature recognition classifies the preprocessed data according to different feature information, and establishes a multi-dimensional user feature information set; a large amount of log data and business data are subjected to machine learning and natural language processing, and multidimensional user portraits are deeply depicted according to characteristic information sets, knowledge maps are constructed, and abnormal transaction risks are identified.
In the embodiment of the invention, the data acquisition mainly comprises terminal trace data (multi-terminal information, multi-device information, network position, software use frequency, software version number, internal version number and the like), service log data (volume of transaction, frequency of transaction, bin holding information and the like) and user behavior information acquired by other modes, and anti-plug-in log data generated by an anti-plug-in system and the like.
And, training the collected data for abnormal transaction detection, the specific detection method may refer to fig. 1, and fig. 1 is a flow chart of the abnormal transaction detection method according to an embodiment of the present invention.
The flow of the abnormal transaction detection method provided by the embodiment of the invention shown in fig. 1 comprises the following steps:
step S101, performing outlier mining analysis on the obtained transaction data to obtain a transaction behavior confidence interval.
In the embodiment of the invention, the transaction behavior confidence interval corresponds to normal transaction data.
Step S102, extracting features of normal transaction data in a transaction behavior confidence interval to obtain transaction behavior features in the transaction behavior confidence interval.
In the embodiment of the invention, the transaction behavior confidence interval comprises transaction behavior characteristics corresponding to normal transaction data. Assuming normal account transaction behavior is a vast majority, abnormal account transaction behavior is a minority. In the overall data sample, abnormal transaction behavior belongs to outliers. The invention adopts an outlier mining algorithm to complete the detection of abnormal behaviors of securities trade.
In particular, outliers are data in the dataset that deviate from confidence intervals, which do not result from random factors, but rather result from entirely different mechanisms. The reasons for the outliers mainly comprise heterogeneous data sources and inherent changes of data variables, are naturally occurring and reflect the data distribution characteristics and data collection errors of the data set (mainly due to human errors, equipment faults or noise and the like). By using an outlier mining algorithm, local abnormal factors in target data can be effectively removed.
In the embodiment of the invention, two targets are mainly realized by adopting an outlier mining algorithm:
(1) Capturing core abnormal factors and rejecting or repairing ineffective abnormal factors:
the anomaly factors represent anomaly data which deviate from confidence intervals after being analyzed by an outlier mining algorithm, wherein the anomaly data are divided into two main categories in the application of the invention: core exception data and invalid exception data.
The core abnormal data is the data which deviates from the confidence interval range, the obtained data is the captured abnormal transaction or abnormal operation and other behaviors, the data is an abnormal factor which is focused by the invention, for example, the abnormal account transaction behavior is assumed to be the vast majority, the intersection of abnormal accounts is the minority, and the captured abnormal account transaction behavior is judged to be the core abnormal factor data after the target data is classified by an outlier mining algorithm.
The invalid abnormal data is similar to the core abnormal data and is an abnormal factor deviating from a confidence interval, but is different from the core abnormal factor, the abnormal data is mainly caused by heterogeneous sources, inherent change of data variables or collection errors, and the abnormal data needs to be removed from target data or artificially repaired to be changed into an effective data set for data training.
(2) Generating a structured dataset:
the invention uses a large amount of semi-structured data and unstructured data in massive logs and business data, the data cannot be directly applied to the invention, and factors which have great influence on the invention deduction in the data need to be removed or repaired, and the process is called as generating a structured data set. And constructing a trusted data set by using the data in the communication interval in the log and the service data by using an outlier mining algorithm, preprocessing the trusted data set, and converting unstructured and semi-structured log and service data into structured data for storage. This data is used to train models of artificial intelligence and generate visual knowledge-maps.
The core abnormal factor data is used as abnormal capturing information in model training, and is used for carrying out differential training with data in a confidence interval to obtain a multi-type knowledge graph, so that the recognition capability of the invention is improved. The invalid abnormal factor data has very small proportion of all data, and can be rejected or manually repaired, the data meeting the requirements is input into an algorithm model again, and the algorithm is iterated for a plurality of times to obtain a dynamic knowledge graph.
Outlier mining is one of the data mining techniques used to study a small portion of data that deviates from most objects. Outlier mining can be described as follows: given a set of n data points or objects, and the expected data k of isolated points, a certain threshold is set, the difference between the objects is quantified by the threshold, and all objects whose differences from the remaining objects exceed the threshold are found. The invention uses the LDC-mine 2 algorithm of isolated point mining, and the following are defined:
definition 1: k distance of object p: for any positive number k, the k-distance of the object p, i.e., k-dis (p), is defined as d (p, o), between the objects p and o, satisfies: at least k objects o 'e D, such that D (p, o'). Ltoreq.d (p, o); there are at most k-1 objects o '∈d, such that D (p, o') < D (p, o).
Definition 2: k-distance neighbor of object p: given the k-distance of an object p, the k-distance of p includes all objects that sum to a distance of p less than the k-distance, namely:
N k-dis (p)＝{q∈D\{p}|d(p,q)}≤k-dis(p)}
these objects are called k-distance neighbors of p.
Definition 3: local deviation rate of object p: giving a circle containing all k-distance neighbors by taking the k-distance neighbor of the object p as a circle center and taking the k-distance as a radius, and calculating Let k be the centroid p' of the neighbor and then the local deviation rate LDR of the object p k (p) is:
the LDC-mini algorithm, in short, considers that the larger the local deviation rate of the object p is, the smaller the influence of the object set on the object p is in a circle with the object p as a center and k as a radius of a distance, the larger the probability that the object p deviates from most objects is, and the more likely the object p is an outlier.
An algorithm based on outlier mining is largely used in the anomaly detection model in the invention. The algorithm is used in a high buying and low selling model, a fund frequent access model, a long-term idle fund frequent transaction model, a entrusted request dos attack model, a query request dos attack model and the like. The physical meaning of each dimension of the k-distance space in each model is different, but the applied algorithm is the same, and the application of the LDC-mini algorithm in the early warning invention is illustrated by taking the entrusted request dos attack model as an example.
The delegated request dos attack refers to: the user sends a large number of entrusting requests by using the account number of the user for a certain purpose, attacks the trading invention of the securities corporation, occupies the resources of the trading invention of the securities corporation and the trading invention, and further influences the normal trading behavior of other users. Two indexes in the delegation request attack are different from the normal delegation request: a "small delegate/total delegate" ratio and a "failed delegate/total delegate" ratio. The step of detecting the entrusting request dos attack by using the LDC algorithm is as follows:
A. Obtaining default objects of 'small commission/total commission' ratio and 'failed commission/total commission' ratio according to historical sample, p d ＝<s d ,f d >。
B. Calculating the 'small commission/total commission' ratio and the 'failure commission/total commission' ratio of the current account in a period of time to obtain an object p c ＝<s c ,f c >。
C. Determination of p c Is a k distance of (c).
D. Retrieval history sample determination p c K-distance neighbor sets of (c).
E. Calculating an object p c Is a local deviation ratio of (2):
if p is c When the LDR exceeds a predetermined threshold, the description object p is described c Belonging to an outlier, the account may be initiating a dos attack of the delegated request.
And step S103, determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data.
As an optional implementation manner, the determining, in step S103, the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data may specifically be:
extracting characteristics of abnormal transaction data deviating from the transaction behavior confidence interval;
identifying the abnormal type of the abnormal transaction feature to obtain the abnormal type corresponding to the abnormal transaction feature; wherein the anomaly types comprise transaction behavior anomaly types and transaction error anomaly types;
if the abnormal type of the abnormal transaction characteristic is the abnormal type of the transaction behavior, determining the abnormal transaction characteristic as a core abnormal transaction characteristic;
And determining the transaction data corresponding to the core abnormal transaction characteristics as target abnormal transaction data.
In the embodiment of the invention, the abnormal factors represent abnormal data which deviate from the confidence interval after being analyzed by an outlier mining algorithm, wherein the abnormal data are divided into two main categories in the application of the invention: core exception data and invalid exception data.
The core abnormal data is the data which deviates from the confidence interval range, the obtained data is the captured abnormal transaction or abnormal operation and other behaviors, the data is an abnormal factor which is focused by the invention, for example, the abnormal account transaction behavior is assumed to be the vast majority, the intersection of abnormal accounts is the minority, and the captured abnormal account transaction behavior is judged to be the core abnormal factor data after the target data is classified by an outlier mining algorithm.
The invalid abnormal data is similar to the core abnormal data and is an abnormal factor deviating from a confidence interval, but is different from the core abnormal factor, the abnormal data is mainly caused by heterogeneous sources, inherent change of data variables or collection errors, and the abnormal data needs to be removed from target data or artificially repaired to be changed into an effective data set for data training.
As an alternative embodiment, after determining the abnormal transaction characteristic as the core abnormal transaction characteristic, the following steps may be further performed:
carrying out full-quantity analysis on the transaction behavior characteristics in the transaction behavior confidence interval, and constructing a structured data set to obtain a first knowledge graph;
performing differential training on the core abnormal transaction characteristics and the transaction behavior characteristics in the transaction behavior confidence interval to obtain a second knowledge graph;
determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs;
model training is continuously carried out through a large amount of user transaction data, and knowledge graph construction is perfected.
As an alternative embodiment, after determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs, the following steps may be further performed:
determining user information in the multi-type knowledge graph;
extracting an associated knowledge graph of each piece of user information from the multi-type knowledge graph;
and obtaining the historical transaction information network relation of each user according to the associated knowledge graph.
Deducing the historical transaction behaviors of each user, mining and identifying abnormal situations in the user transaction behaviors, and predicting abnormal events and occurrence probability of the predicted abnormal events according to the existing data of the user;
If the occurrence probability is larger than a preset probability threshold, generating and outputting abnormal event early warning information based on the predicted abnormal event corresponding to the occurrence probability.
In the embodiment of the invention, a large amount of semi-structured data and unstructured data exist in the used massive logs and business data, the data cannot be directly applied to a system, and factors which have great influence on system deduction in the data need to be removed or repaired, and the process is called as generating a structured data set. And constructing a trusted data set by using the data in the communication interval in the log and the service data by using an outlier mining algorithm, preprocessing the trusted data set, and converting unstructured and semi-structured log and service data into structured data for storage. This data is used to train models of artificial intelligence and generate visual knowledge-maps.
The core abnormal factor data is used as abnormal capturing information in model training, and is used for carrying out differential training with data in a confidence interval to obtain a multi-type knowledge graph, so that the recognition capability of the system is improved. The invalid abnormal factor data has very small proportion of all data, and can be rejected or manually repaired, the data meeting the requirements is input into an algorithm model again, and the algorithm is iterated for a plurality of times to obtain a dynamic knowledge graph.
Knowledge graph combines theory and method of application mathematics, graphics, information visualization technology, information science and other subjects with methods of metering introduction analysis, co-occurrence analysis and the like, and the core structure is displayed visually by utilizing the visualized graph. Knowledge graph has undergone several stages of development from two-dimensional graph, three-dimensional configuration, multidimensional scale graph, social network analysis graph, self-organizing map graph, path finding network graph, etc.
The traditional risk identification technology has the defects of incomplete data coverage, unclear vein carding, non-real-time risk monitoring and the like, and needs a large amount of manual participation to complete wind control analysis, the timeliness and accuracy of the assessment depend on participants excessively, and accurate risk assessment cannot be given.
According to the invention, the dynamic knowledge graph is introduced, the adjacent associated networks are obtained from the mass storage data, and dynamically evolve along with time, so that the network relationship in the complex data is more intuitively displayed, the potential risk information is favorably found, and effective associated evidence is provided for the case. The algorithm used by the system mainly comprises the functions of information integration, relationship identification, risk identification, early warning and the like.
(1) Information integration: the log data and the business data of different types have respective text formats and storage modes, the traditional solution cannot consider global data characteristics, only analysis is carried out aiming at single data characteristics, the resource utilization rate is low, and the information dependency relationship cannot be effectively and comprehensively analyzed. And loading the scattered information by using a dynamic knowledge graph algorithm, performing full-quantity analysis on the information, and extracting a correlation topology network between the data to achieve a transverse and longitudinal multidimensional analysis mode.
(2) And (3) relationship identification: after information integration, a large number of data sources are input through a neural network to perform model training, and knowledge maps of respective users are extracted from integrated structured data. The individual is taken as an independent individual to participate in the transaction, and the network structure constructed by the individual is relatively simple and has no huge association relation. The mechanism is taken as a core market participant, has a large number of relation networks and organization relations, and knows the 'previous generation today' of each participating entity through the corresponding relation state among knowledge graph association organizations. The construction of the network relationship in the dynamic knowledge graph shows the evolution of the relationship network between each entity and restores the real history of the entity along with the time.
(3) Risk identification and early warning: the relation recognition of the dynamic knowledge graph forms a huge entity relation network, the historical situation of each entity is deduced through an algorithm, potential abnormal situations in the history are analyzed according to the current policy, violation information in the history is mined and recognized, and the abnormal situations appearing in calculation history data are visualized and displayed. After the knowledge graph calculates the historical information of each entity, whether each entity has potential safety hazards or not is identified, whether an event occurs or not is calculated according to the existing data, and early warning information is given.
The invention uses the deep learning model to build the abnormal transaction detection model input for the user data time division, personnel division and grading, so that the model learns the association and finds out the abnormal log more accurately. The system uses a RAdam optimizer to provide a dynamic heuristic to provide automated variance reduction, eliminating the need for manual tuning involved in wakeup during training. The learning rate determines how fast the parameter moves to the optimal value, and if the learning rate is too high, the parameter is likely to cross the optimal value; conversely, if the learning rate is too small, the efficiency of the optimization may be too low and the algorithm may not converge for a long period of time. The platform automatically reduces and adjusts different learning rates according to the fact that the learning rate is not improved in 3 epochs. The abnormal transaction detection model is shown in fig. 3. Specific:
(1) All fields in the log information are used as characteristics, time is used as a position to be encoded, and the model is input.
(2) Self-Attention: from the attention network, a large number of experimental verification can better assist in learning the association among various information, and the key point of the model is to reconstruct the model.
(3) And the model multichannel feature transformation is added similarly to a residual network module, so that the gradient vanishing risk is reduced.
(4) And the scoring module is used for adding a global view and assisting in checking whether abnormal operation exists in the input, if not, the scoring module is directly used for scoring 0, and the model fitting is accelerated.
(5) The data volume is larger, the complexity of the model is increased, and the overfitting is avoided.
The epoch refers to that the model is completely trained once by using all data of the training set, and the effect graph (the abscissa is the epoch number) shown in fig. 4 is obtained by carrying out 100 epochs on the training set by the platform.
The ratio of the number of correctly classified samples to the total number of samples of the model is calculated by the accuracy of the abnormal transaction detection model to measure the effect of the model, with the goal of measuring the effect of the model. Model parameters are updated through loss calculation of the model, and the aim is to reduce optimization errors, namely experience risks of the model are reduced under the combined action of a loss function and an optimization algorithm. As epoch increases, the accuracy of the model increases and the loss value decreases. The abnormal transaction detection model operation effect diagram is shown in fig. 4.
In the embodiment of the invention, after depth, machine learning and rule processing, all abnormal information is stored in the database, and if the data volume is huge, the data can be stored in a large data platform. Since the abnormality identified by the system cannot ensure hundred percent accuracy, the system identification efficiency may be lowered once a new problem occurs. In order to avoid the recognition error, after the machine recognizes the abnormal log, the professional personnel review the abnormal log, and the data after review is reintroduced into the database to form a feedback mechanism. The system is improved from each feedback, and the iteration is continuously perfected, so that the big data analysis capability of the whole system is improved. The database data is then exported in a desired form, such as a data report or web page presentation interface.
The invention can analyze the transaction data and accurately identify abnormal transactions.
In the embodiment of the invention, the abnormal transaction data can be investor abnormal transaction data, and the investor abnormal transaction behavior mainly comprises the following types: false declaration, pulling up and pressing stock prices, maintaining limit price of rising (falling) amplitude, self-buying and self-selling or trading against each other, severely abnormally fluctuating stock declaration rate abnormality, and other abnormal trading actions that violate laws, administrative laws, department regulations, normative documents, or deep-exchange business rules. Specific:
(1) Behavior characteristics of corresponding abnormal transaction data are declared in false mode: the investor declares that the amount is larger, and the revocation declaration accounts for higher; the number of the strands is small or the ratio of the strands is low, and the purpose of the strand is not to be achieved. Typical false declaration actions include:
false declarations during open set bidding. The method mainly refers to the behavior of influencing the offer through high price declaration and withdrawal declaration during the period of 9:15-9:20 bid of the offer set to permit withdrawal of the order. During this period, individual investors first conduct buying declaration at a price significantly higher than the release price, which leads to rapid rising of virtual open price revealed by quotation, attracting other investors to buy with the wind, and then cancel all buying declarations themselves, and finally possibly forming open price exceeding the market expectations.
False declaration in the continuous bidding phase. The method mainly refers to the continuous bidding stage, and the buying is declared in the effective bidding range revealed by the quotation through the large-stroke declaration and continuous declaration means, and then the behavior of withdrawing the form is adopted. This behaviour is prone to the illusion of active buying discs, misleading other investors to judge the stock price trend, and inducing investors, especially small and medium-sized scattered households, to buy with the wind.
(2) Behavior characteristics of abnormal transaction data corresponding to pulling, lifting and pressing: investors buy or sell related securities at large prices or frequently declared to be significantly off the latest bid price, common including:
high price reporting, large pen reporting and huge amount reporting during open set bidding. Mainly refers to the bidding period of the opening set, particularly in the irremovable single stage of 9:20-9:25, the buying is declared in a large quantity at a price which is obviously higher than the virtual opening price, and thus the opening price is influenced or even determined. This behavior is often the main determining force for the first day of new stock market.
And high price declaration in the continuous bidding stage. The continuous bidding stage after the start of the bidding is mainly characterized in that the continuous bidding stage is used for buying at a price which is obviously higher than the latest price of the market revelation, so that the stock price is greatly increased in a short time, other investors are attracted and induced to buy with wind, and the first day of the new stock is easily induced and promoted to be on the market for stir-frying.
(3) Maintaining behavior characteristics of abnormal transaction data corresponding to the limit price of the rising (falling) amplitude: investors limit the price to be declared in large or continuous declaration by the rise or fall, so that the price of the related securities reaches the rising stop or falling stop.
(4) Behavior characteristics of self-buying, self-selling or mutual counter-party transaction corresponding abnormal transaction data:
the single or more than two fixed or suspected related securities accounts are subjected to a large number or frequent reverse transactions, so that the securities transaction price or the transaction amount is influenced; the number of the self-buying and self-selling or mutual counter-party transactions is higher than the total daily transaction number of the strand; stock prices or trade volume are greatly affected.
In addition, common investors' abnormal trading actions include lending securities accounts, using plug-ins, off-site funding, etc.:
lending individual securities accounts may enable illegal personnel to evade and supervise illegal securities activities such as inner screen trade, market manipulation, etc., and illegal financial activities such as suspected money laundering, etc., and are also one of the main means of off-site fund distribution, seriously affecting securities market order and financial security.
The use of plug-in refers to the act of investors illegally accessing a stock company trading system or improperly using a trading system that is not officially issued by stock companies to conduct stock exchanges. The external producer realizes specific purposes (such as conversion into an external interface) for investors by modifying the transaction terminal program and data, and falsifies the original normal setting and rules of the transaction terminal to widen the application range of the transaction terminal, thereby disturbing the market order. The plug-in program can provide a channel for plug-in derivative software (namely, provide external connection in a so-called trade interface form), so that the plug-in is used as a bridge illegal security service form (such as fund distribution, automatic new platform, etc.).
The off-site fund distribution refers to a unit or a person who does not acquire the qualification of the securities business, and takes the profit as the purpose, borrows funds with the proportion of a plurality of times higher than the guarantee fund to the party using the securities to conduct securities trade, ensures the fund distribution not to be lost by collecting the guarantee fund, enjoying the flat right and the like, and earns activities of interest, expense or income division. The off-site allocation forms are similar to loans, but are really money-borrowing and stock-frying for others, and essentially belong to securities financing business. The common off-site fund distribution mainly comprises a plurality of modes of lending accounts, system sub-bins, virtual fraud and the like, and the illegal camping securities business which is not approved by relevant national authorities disturbs the market order, is not protected by law, and is difficult to guarantee the rights and interests of investors.
And (3) characteristic analysis of off-site fund distribution:
1. offline fund distribution
Offline funding is a traditional funding form, since it is usually smaller in scale mainly through offline modes. The traditional offline fund distribution mainly takes a lending account as a main mode, so typical characteristics comprise multiple devices (which can be judged through corresponding device information of the account, such as an equipment name (IP) MAC (media access control)), periodic access and activation of the account, and the like, and the devices and access areas are greatly different each time. When off-line funding exists in a form of a lending account number and a daily market-staring flat warehouse demand exists, the demand exists in an automatic demand, and the demand can be completed through plug-in.
2. Online funding
The online distribution is an advanced mode of offline distribution, which breaks through the limitations of less acquisition, slow turnover, regional operation and the like of the original offline distribution operation at one time, so that the online distribution has great hazard. Because securities companies do not provide access channels for illegal securities business operations, plug-in access is a possible way of online funding.
By combining the transaction abnormal behavior feature analysis with industry experience, the method and the system for detecting the abnormal transaction behavior based on the user basic data, the transaction behavior data, the fund data, the anti-plug-in data and other data are provided by applying an artificial intelligence technology and an anti-plug-in technology. The technology of informatization, systemization, big data, artificial intelligence and the like is utilized to collect originally scattered data and experiences so as to carry out system operation. The manual operation flow is converted into the machine realization flow, and the discovery capability and accuracy of abnormal transaction behaviors are improved.
Having described the method of an exemplary embodiment of the present invention, an abnormal transaction detection system of an exemplary embodiment of the present invention is described next with reference to fig. 2, the system comprising:
the analysis unit 201 is configured to perform outlier mining analysis on the obtained transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data;
An extracting unit 202, configured to perform feature extraction on the normal transaction data in the transaction behavior confidence interval obtained by the analyzing unit 201, so as to obtain transaction behavior features in the transaction behavior confidence interval;
a determining unit 203 for determining the transaction data deviating from the transaction behavior confidence interval obtained by the analyzing unit 201 as abnormal transaction data.
In the embodiment of the invention, the abnormal transaction detection system can gather originally scattered data and experiences by utilizing the technologies of informatization, systemization, big data, artificial intelligence and the like to perform system operation. The manual operation flow is converted into the machine realization flow, and the discovery capability and accuracy of abnormal transaction behaviors are improved.
Optionally, the abnormal transaction detection system mainly includes: the data layer (based on data cleaning and analysis of logs, business and the like, forming a structured real-time digital bin), the algorithm layer (big data, machine learning, knowledge graph, natural language processing) and the application layer (anti-plug-in, user portrait, anomaly detection).
The data is a foundation, and an enormous data warehouse is constructed by gathering historical log information and business data information according to the support of industrial data; the algorithm is a core, a data warehouse is analyzed by utilizing big data and an artificial intelligence advanced technology, a structured storage data set is obtained, and a rich index library and a model library are established; the application is a valuable embodiment, and the model is converted into a product and applied to intelligent wind control.
High-efficiency big data technology is introduced, and the method is applied to data cleaning, data storage, data modeling and data application of huge data warehouse. By utilizing a data mining technology, a big data abnormal transaction analysis model is established, data samples required by intelligent abnormal transactions are analyzed, data cleaning, storage, modeling and application are performed in real time, service data and log data are applied in real time, the problem of information asymmetry is solved, and a high-quality, diversified and multi-dimensional sample model is provided for the system.
The four major techniques of artificial intelligence application core include feature recognition, machine learning, natural language processing, knowledge graph. The feature recognition classifies the preprocessed data according to different feature information, and establishes a multi-dimensional user feature information set; a large amount of log data and business data are subjected to machine learning and natural language processing, and multidimensional user portraits are deeply depicted according to characteristic information sets, knowledge maps are constructed, and abnormal transaction risks are identified.
As an alternative embodiment, the determining unit 203 determines the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data specifically includes:
extracting characteristics of abnormal transaction data deviating from the transaction behavior confidence interval;
Identifying the abnormal type of the abnormal transaction feature to obtain the abnormal type corresponding to the abnormal transaction feature; wherein the anomaly types comprise transaction behavior anomaly types and transaction error anomaly types;
if the abnormal type of the abnormal transaction characteristic is the abnormal type of the transaction behavior, determining the abnormal transaction characteristic as a core abnormal transaction characteristic;
and determining the transaction data corresponding to the core abnormal transaction characteristics as target abnormal transaction data.
As an alternative embodiment, the determining unit 203 is further configured to:
after the abnormal transaction characteristics are determined to be core abnormal transaction characteristics, carrying out full-quantity analysis on the transaction behavior characteristics in the transaction behavior confidence interval, and constructing a structured data set to obtain a first knowledge graph;
performing differential training on the core abnormal transaction characteristics and the transaction behavior characteristics in the transaction behavior confidence interval to obtain a second knowledge graph;
determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs;
model training is continuously carried out through a large amount of user transaction data, and knowledge graph construction is perfected.
As an alternative embodiment, the determining unit 203 is further configured to:
After the first knowledge-graph and the second knowledge-graph are determined to be multi-type knowledge-graphs, determining user information in the multi-type knowledge-graphs;
extracting an associated knowledge graph of each piece of user information from the multi-type knowledge graph;
and obtaining the historical transaction information network relation of each user according to the associated knowledge graph.
As an alternative embodiment, the determining unit 203 is further configured to:
after obtaining the historical transaction information network relation of each user, deducing the historical transaction behavior of each user, mining and identifying abnormal situations in the transaction behavior of the user, and predicting abnormal events and occurrence probability of the predicted abnormal events according to the existing data of the user;
if the occurrence probability is larger than a preset probability threshold, generating and outputting abnormal event early warning information based on the predicted abnormal event corresponding to the occurrence probability.
In the description of the present invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Furthermore, although the operations of the methods of the present invention are depicted in the drawings in a particular order, this is not required to either imply that the operations must be performed in that particular order or that all of the illustrated operations be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
Claims (10)
1. An abnormal transaction detection method, comprising:
performing outlier mining analysis on the obtained transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data;
extracting features of normal transaction data in a transaction behavior confidence interval to obtain transaction behavior features in the transaction behavior confidence interval;
and determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data.
2. The abnormal transaction detection method according to claim 1, the determining transaction data deviating from the transaction behavior confidence interval as abnormal transaction data, comprising:
extracting characteristics of abnormal transaction data deviating from the transaction behavior confidence interval;
Identifying the abnormal type of the abnormal transaction feature to obtain the abnormal type corresponding to the abnormal transaction feature; wherein the anomaly types comprise transaction behavior anomaly types and transaction error anomaly types;
if the abnormal type of the abnormal transaction characteristic is the abnormal type of the transaction behavior, determining the abnormal transaction characteristic as a core abnormal transaction characteristic;
and determining the transaction data corresponding to the core abnormal transaction characteristics as target abnormal transaction data.
3. The abnormal transaction detection method of claim 2, after determining the abnormal transaction signature as a core abnormal transaction signature, the method further comprising:
carrying out full-quantity analysis on the transaction behavior characteristics in the transaction behavior confidence interval, and constructing a structured data set to obtain a first knowledge graph;
performing differential training on the core abnormal transaction characteristics and the transaction behavior characteristics in the transaction behavior confidence interval to obtain a second knowledge graph;
determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs; model training is continuously carried out through a large amount of user transaction data, and knowledge graph construction is perfected.
4. The abnormal transaction detection method according to claim 3, after determining the first knowledge-graph and the second knowledge-graph as a multi-type knowledge-graph, the method further comprising:
determining user information in the multi-type knowledge graph;
extracting an associated knowledge graph of each piece of user information from the multi-type knowledge graph;
and obtaining the historical transaction information network relation of each user according to the associated knowledge graph.
5. The abnormal transaction detection method according to claim 4, wherein after the obtaining the historical transaction information network relationship of each user, the method further comprises:
deducing the historical transaction behaviors of each user, mining and identifying abnormal situations in the user transaction behaviors, and predicting abnormal events and occurrence probability of the predicted abnormal events according to the existing data of the user;
if the occurrence probability is larger than a preset probability threshold, generating and outputting abnormal event early warning information based on the predicted abnormal event corresponding to the occurrence probability.
6. An abnormal transaction detection system, comprising:
the analysis unit is used for carrying out outlier mining analysis on the acquired transaction data to obtain a transaction behavior confidence interval; wherein, the transaction behavior confidence interval corresponds to normal transaction data;
The extraction unit is used for extracting the characteristics of the normal transaction data in the transaction behavior confidence interval to obtain the transaction behavior characteristics in the transaction behavior confidence interval;
and the determining unit is used for determining the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data.
7. The abnormal transaction detection system according to claim 6, wherein the determination unit determines the transaction data deviating from the transaction behavior confidence interval as abnormal transaction data in a manner that:
extracting characteristics of abnormal transaction data deviating from the transaction behavior confidence interval;
identifying the abnormal type of the abnormal transaction feature to obtain the abnormal type corresponding to the abnormal transaction feature; wherein the anomaly types comprise transaction behavior anomaly types and transaction error anomaly types;
if the abnormal type of the abnormal transaction characteristic is the abnormal type of the transaction behavior, determining the abnormal transaction characteristic as a core abnormal transaction characteristic;
and determining the transaction data corresponding to the core abnormal transaction characteristics as target abnormal transaction data.
8. The abnormal transaction detection system of claim 7, the determination unit further configured to:
After the abnormal transaction characteristics are determined to be core abnormal transaction characteristics, carrying out full-quantity analysis on the transaction behavior characteristics in the transaction behavior confidence interval, and constructing a structured data set to obtain a first knowledge graph;
performing differential training on the core abnormal transaction characteristics and the transaction behavior characteristics in the transaction behavior confidence interval to obtain a second knowledge graph;
determining the first knowledge-graph and the second knowledge-graph as multiple types of knowledge-graphs; model training is continuously carried out through a large amount of user transaction data, and knowledge graph construction is perfected.
9. The abnormal transaction detection system of claim 8, the determination unit further to:
after the first knowledge-graph and the second knowledge-graph are determined to be multi-type knowledge-graphs, determining user information in the multi-type knowledge-graphs;
extracting an associated knowledge graph of each piece of user information from the multi-type knowledge graph;
and obtaining the historical transaction information network relation of each user according to the associated knowledge graph.
10. The abnormal transaction detection system of claim 9, the determination unit further configured to:
after obtaining the historical transaction information network relation of each user, deducing the historical transaction behavior of each user, mining and identifying abnormal situations in the transaction behavior of the user, and predicting abnormal events and occurrence probability of the predicted abnormal events according to the existing data of the user;
If the occurrence probability is larger than a preset probability threshold, generating and outputting abnormal event early warning information based on the predicted abnormal event corresponding to the occurrence probability.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310953489.3A CN117131445A (en) | 2023-07-28 | 2023-07-28 | Abnormal transaction detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310953489.3A CN117131445A (en) | 2023-07-28 | 2023-07-28 | Abnormal transaction detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117131445A true CN117131445A (en) | 2023-11-28 |
Family
ID=88857493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310953489.3A Pending CN117131445A (en) | 2023-07-28 | 2023-07-28 | Abnormal transaction detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117131445A (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7974905B1 (en) * | 2008-07-15 | 2011-07-05 | Paul Chi | Outlier trade detection for securities lending transactions |
| US20200073932A1 (en) * | 2018-08-30 | 2020-03-05 | Intelligent Fusion Technology, Inc | Method and system for pattern discovery and real-time anomaly detection based on knowledge graph |
| US20200081445A1 (en) * | 2018-09-10 | 2020-03-12 | Drisk, Inc. | Systems and Methods for Graph-Based AI Training |
| US20200334680A1 (en) * | 2019-04-22 | 2020-10-22 | Paypal, Inc. | Detecting anomalous transactions using machine learning |
| CN112787841A (en) * | 2019-11-11 | 2021-05-11 | 华为技术有限公司 | Fault root cause positioning method and device and computer storage medium |
| CN114140248A (en) * | 2021-12-14 | 2022-03-04 | 重庆忽米网络科技有限公司 | AI artificial intelligence technology-based abnormal transaction identification method |
| CN114265835A (en) * | 2021-10-28 | 2022-04-01 | 深圳永安在线科技有限公司 | Data analysis method and device based on graph mining and related equipment |
| CN114580861A (en) * | 2022-02-17 | 2022-06-03 | 珠海格力电器股份有限公司 | Energy data anomaly detection method and device and energy system |
| CN114781715A (en) * | 2022-04-17 | 2022-07-22 | 北京云集智造科技有限公司 | Anomaly recognition device based on big data and artificial intelligence |
| CN114971642A (en) * | 2022-05-30 | 2022-08-30 | 平安科技（深圳）有限公司 | Knowledge graph-based anomaly identification method, device, equipment and storage medium |
| CN115471309A (en) * | 2022-08-12 | 2022-12-13 | 北京神州新桥科技有限公司 | Transaction abnormity detection method and device, electronic equipment and readable storage medium |
| CN115641201A (en) * | 2022-09-27 | 2023-01-24 | 厦门国际银行股份有限公司 | Data anomaly detection method, system, terminal device and storage medium |
| CN115828756A (en) * | 2022-12-09 | 2023-03-21 | 西安电子科技大学 | Phased array antenna excitation inversion method based on design and fault knowledge graph |
| US20230124411A1 (en) * | 2021-10-15 | 2023-04-20 | Blackrock, Inc. | Machine learning engine for determining data similarity |
| CN116012152A (en) * | 2022-09-09 | 2023-04-25 | 成都农村商业银行股份有限公司 | Method, device and equipment for identifying abnormal transaction entity and readable storage medium |
| WO2023097026A2 (en) * | 2021-11-23 | 2023-06-01 | Strong Force TX Portfolio 2018, LLC | Transaction platforms where systems include sets of other systems |
| US20230186120A1 (en) * | 2021-11-24 | 2023-06-15 | Intelligent Fusion Technology, Inc. | Methods and systems for anomaly and pattern detection of unstructured big data |
-
2023
- 2023-07-28 CN CN202310953489.3A patent/CN117131445A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7974905B1 (en) * | 2008-07-15 | 2011-07-05 | Paul Chi | Outlier trade detection for securities lending transactions |
| US20200073932A1 (en) * | 2018-08-30 | 2020-03-05 | Intelligent Fusion Technology, Inc | Method and system for pattern discovery and real-time anomaly detection based on knowledge graph |
| US20200081445A1 (en) * | 2018-09-10 | 2020-03-12 | Drisk, Inc. | Systems and Methods for Graph-Based AI Training |
| US20200334680A1 (en) * | 2019-04-22 | 2020-10-22 | Paypal, Inc. | Detecting anomalous transactions using machine learning |
| CN112787841A (en) * | 2019-11-11 | 2021-05-11 | 华为技术有限公司 | Fault root cause positioning method and device and computer storage medium |
| US20230124411A1 (en) * | 2021-10-15 | 2023-04-20 | Blackrock, Inc. | Machine learning engine for determining data similarity |
| CN114265835A (en) * | 2021-10-28 | 2022-04-01 | 深圳永安在线科技有限公司 | Data analysis method and device based on graph mining and related equipment |
| WO2023097026A2 (en) * | 2021-11-23 | 2023-06-01 | Strong Force TX Portfolio 2018, LLC | Transaction platforms where systems include sets of other systems |
| US20230186120A1 (en) * | 2021-11-24 | 2023-06-15 | Intelligent Fusion Technology, Inc. | Methods and systems for anomaly and pattern detection of unstructured big data |
| CN114140248A (en) * | 2021-12-14 | 2022-03-04 | 重庆忽米网络科技有限公司 | AI artificial intelligence technology-based abnormal transaction identification method |
| CN114580861A (en) * | 2022-02-17 | 2022-06-03 | 珠海格力电器股份有限公司 | Energy data anomaly detection method and device and energy system |
| CN114781715A (en) * | 2022-04-17 | 2022-07-22 | 北京云集智造科技有限公司 | Anomaly recognition device based on big data and artificial intelligence |
| CN114971642A (en) * | 2022-05-30 | 2022-08-30 | 平安科技（深圳）有限公司 | Knowledge graph-based anomaly identification method, device, equipment and storage medium |
| CN115471309A (en) * | 2022-08-12 | 2022-12-13 | 北京神州新桥科技有限公司 | Transaction abnormity detection method and device, electronic equipment and readable storage medium |
| CN116012152A (en) * | 2022-09-09 | 2023-04-25 | 成都农村商业银行股份有限公司 | Method, device and equipment for identifying abnormal transaction entity and readable storage medium |
| CN115641201A (en) * | 2022-09-27 | 2023-01-24 | 厦门国际银行股份有限公司 | Data anomaly detection method, system, terminal device and storage medium |
| CN115828756A (en) * | 2022-12-09 | 2023-03-21 | 西安电子科技大学 | Phased array antenna excitation inversion method based on design and fault knowledge graph |
Non-Patent Citations (3)
| Title |
|---|
| PEIZHENG HUANG 等: "Overview of the Application of Knowledge Graph in Anomaly Detection and Fault Diagnosis", 《2022 4TH INTERNATIONAL CONFERENCE ON SYSTEM RELIABILITY AND SAFETY ENGINEERING》, 31 December 2022 (2022-12-31) * |
| 王海波: "基于知识图谱并考虑智能数据关联的金融反洗钱软件设计及应用", 《软件导刊》, vol. 22, no. 1, 31 January 2023 (2023-01-31) * |
| 郑杰: "试验设计与数据分析[M]", 31 August 2016, 《广州：华南理工大学出版社》, pages: 120 - 123 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109347801B (en) | Vulnerability exploitation risk assessment method based on multi-source word embedding and knowledge graph | |
| CN109544163B (en) | Risk control method, device, equipment and medium for user payment behavior | |
| Siering et al. | A taxonomy of financial market manipulations: establishing trust and market integrity in the financialized economy through automated fraud detection | |
| KR102032924B1 (en) | Security System for Cloud Computing Service | |
| KR102058697B1 (en) | Financial fraud detection system by deeplearning neural-network | |
| CN112116464B (en) | Abnormal transaction behavior analysis method and system based on event sequence frequent item set | |
| KR102618473B1 (en) | Analyzing news contents service system of cryptocurrency using robot journalism | |
| Li et al. | Theory and application of artificial intelligence in financial industry | |
| CN111369344A (en) | Method and device for dynamically generating early warning rule | |
| Adebiyi | Exploring the impact of predictive analytics on accounting and auditing expertise: A regression analysis of LinkedIn survey data | |
| Min et al. | Behavior language processing with graph based feature generation for fraud detection in online lending | |
| CN112950359B (en) | User identification method and device | |
| CN112669039B (en) | Knowledge graph-based customer risk management and control system and method | |
| CN113269378A (en) | Network traffic processing method and device, electronic equipment and readable storage medium | |
| Wang | Research on bank marketing behavior based on machine learning | |
| CN117131445A (en) | Abnormal transaction detection method and system | |
| CN113055368B (en) | Web scanning identification method and device and computer storage medium | |
| CN115033891A (en) | Vulnerability assessment method and device, storage medium and electronic equipment | |
| CN114723548A (en) | Data processing method, apparatus, device, medium, and program product | |
| CN114066624A (en) | Blacklist associated map mining method and system based on graph calculation | |
| CN112712270A (en) | Information processing method, device, equipment and storage medium | |
| CN114039837A (en) | Alarm data processing method, device, system, equipment and storage medium | |
| CN114880369A (en) | Risk credit granting method and system based on weak data technology | |
| CN113298641A (en) | Integrity degree cognition method and device | |
| CN115438882A (en) | Business dynamic sensing system based on full-flow monitoring technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |