CN107908300B - A kind of synthesis of user's mouse behavior and analogy method and system - Google Patents

A kind of synthesis of user's mouse behavior and analogy method and system Download PDF

Info

Publication number
CN107908300B
CN107908300B CN201711148377.1A CN201711148377A CN107908300B CN 107908300 B CN107908300 B CN 107908300B CN 201711148377 A CN201711148377 A CN 201711148377A CN 107908300 B CN107908300 B CN 107908300B
Authority
CN
China
Prior art keywords
mouse
point
click
mobile
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711148377.1A
Other languages
Chinese (zh)
Other versions
CN107908300A (en
Inventor
胡书杰
柏军
王佰玲
黄俊恒
王巍
辛国栋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology Weihai
Original Assignee
Harbin Institute of Technology Weihai
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology Weihai filed Critical Harbin Institute of Technology Weihai
Priority to CN201711148377.1A priority Critical patent/CN107908300B/en
Publication of CN107908300A publication Critical patent/CN107908300A/en
Application granted granted Critical
Publication of CN107908300B publication Critical patent/CN107908300B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03543—Mice or pucks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45579—I/O management, e.g. providing access to device drivers or storage

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present invention relates to a kind of synthesis of user's mouse behavior and analogy method and system, comprising: (1) data acquisition and procession: the original mouse data of acquisition user in real time, after segmentation, cleaning, is stored in historical operating data library；(2) model training: for without mobile operation, training transversal normal distribution model generates time interval parameter；For needing mobile operation, cluster feature and matching characteristic are extracted, training set is constructed；(3) operation synthesis and simulation: for using the message sequence of transversal normal distribution Construction of A Model simulated operation without mobile operation；For needing mobile operation to match a sample as template using Clustering Model and disaggregated model；Synthesize the synthetic operation sequence for meeting the requirement parameter to simulated operation.

Description

A kind of synthesis of user's mouse behavior and analogy method and system
Technical field
The present invention relates to a kind of synthesis of user's mouse behavior and analogy method and system, belong to customer behavior modeling technology Field.
Background technique
The effect of mouse Behavior modeling technology is to generate mouse track sequence in various scene Imitatings, and corresponding soft Simulation generates mouse action behavior in hardware platform.Existing mouse emulation technology only stays in generation and does not have human behavior feature Mouse action or duplicate mouse action is generated by simple data readback or converts mould for the contact action of user Quasi- mouse action etc. cannot freely generate generated data and the simulation behaviour of the mouse behavioural characteristic with real user Make.In the gradually popular today of the various identification technologies based on mouse behavioural characteristic, these analogy methods can not expansiblely The data for having real user feature are generated, the detection of the existing behavior identifying code based on mouse track can not be passed through.
In recent years, the mouse behavioral data of user is increasingly being used for the authentication of Behavior-based control or biological characteristic With the technical fields such as identification and behavior identifying code based on mouse track.These technologies are based on collected mouse action number According to feature is extracted, with the methods of machine learning train classification models, legitimate user and illegal user or true are distinguished to reach The purpose of real user and robotic user.On the one hand, these technologies are had been able in some scenarios to true user's mouse Operation reaches higher accuracy of identification, such as the picture mosaic identifying code based on mouse drag, in addition to completing result verification according to picture mosaic Except user's authenticity, it can also be judged according to the dynamic characteristic of mouse action track, therefore traditional direct simulation mouse Operating method, as at the uniform velocity or the mobile analogy method of the mouse of even acceleration under these identification technologies without be stealthy, Wu Fatong Cross detection.On the other hand, the practical application of these identification technologies has very big requirement to data volume, and being completely dependent on truthful data can drop The availability of low technical；Moreover, effective mouse Behavior modeling data still can be generated without methodology at present, to these The safety and reliability that technology resists simulation attack is verified and is tested.
Summary of the invention
In view of the deficiencies of the prior art, the present invention provides the synthesis and analogy method of a kind of user's mouse behavior；
The present invention also provides the synthesis and simulation system of a kind of user's mouse behavior；
The present invention can realize in the case where given certain user on a small quantity true mouse action data to any common mouse The synthesis and simulation of operation behavior, it is special that the mouse action behavior for synthesizing and simulating will be provided with certain original subscriber's mouse behavior Sign.
The automation that the present invention can be used in mouse behavior associated verification code cracks, the various statistics based on mouse behavior Practise reinforcing, the various models based on user's mouse behavior and the performance test of system etc. of model.
The technical solution of the present invention is as follows:
A kind of synthesis and analogy method of the behavior of user's mouse, comprising:
(1) data acquisition and procession
The original mouse data of user are acquired in real time, and original mouse data refer to (x, y, t, action) four-tuple, that is, mouse The original mouse sequence of operation that operating point or mouse information point are constituted is marked, (x, y) is the coordinate of mouse on the screen, and unit is picture Vegetarian refreshments；T is the timestamp that mouse action generates, and unit is millisecond or microsecond；Action is the type of mouse action, including mouse It is mobile, idler wheel scrolls up, left button is pressed or bounce, right button is pressed or bounce, middle key pressing or bounce and other Function key pressing is bounced；
Specific operation will be converted into after original mouse data manipulation segmentation, specific operation include left button click, right button Click, left double click after left button is clicked, moved after left double click, middle key are clicked, other function key is clicked, moved, it is mobile after it is right Key is clicked, idler wheel continuous rolling, left button pull, right button pulls, then is filtered out via data cleansing by system call or user's mistake After the abnormal data caused by reasons such as touching, it is stored in historical operating data library；
(2) model training
For including that left button is clicked, right button is clicked, left double click, middle key without mobile operation without mobile operation Click, other function key is clicked, idler wheel continuous rolling, training transversal normal distribution model, by its probability distribution generation the time between Every parameter, time interval parameter is the time interval between each mouse information, e.g., single-click operation: key mapping presses message, key Bounce the time interval between message in position；
For the mobile operation of needs, left double click, shifting after needing mobile operation to click, move including left button after movement Right button is clicked after dynamic, left button pulls, right button pulls, and extracts cluster feature to the data in historical operating data library and matching is special Sign constructs training set；Cluster feature clusters each category for training Clustering Model, using cluster result as The second level class label of historical operation sample；Matching characteristic combination Clustering Model is used for the second level class label of sample labeling To one disaggregated model of each category operation training containing moving operation, for not knowing institute still under the category Belong to the operation sample of second level classification, which is classified as under the category according to the matching characteristic vector of the sample A specific second level classification, to match the higher template samples of similarity (under be referred to as template samples)；Category Be it is above-mentioned clicked through system pretreatment come the left button that sorts out, left double click, it is mobile after left button is clicked, left button pulls, second level Classification is voluntarily to cluster the classification of generation according to data sample under each specific category by clustering algorithm；Such as: to movement The category sample that left button is clicked afterwards is clustered, and generates several sample clusters, left button clicks one after the as movement of each cluster A second level classification (such as: single-click operation after the first kind is mobile, single-click operation, etc. after the second class is mobile) under grade classification, this A little clusters are automatically generated by Clustering Model algorithm according to cluster feature, do not need engineer's sub-clustering rule.
The model for one generic operation (calling category in the following text) being clustered, being subdivided into second level classification, with the model to such Sample under operation carries out classification (calling second level classification in the following text) label in class；The matching characteristic is in cluster feature by simulation demand The character subset of release, it is sub for matching a second level classification under the requirement for being intended to simulated operation, then by the second level classification sample Collection is for matching specific historical data target (i.e. template samples)；
(3) operation synthesis and simulation
One behavior based on mouse action can be decomposed into the combination of several category mouse actions, and successively simulation is every The simulation of mouse behavior can be realized in a category mouse action.
For being generated needed for simulated operation using trained transversal normal distribution model is given without mobile operation The message sequence that time interval, constructing analog operate；
For needing mobile operation, (wherein starting point can be defaulted and make for the mobile starting point coordinate of input and terminal point coordinate With mouse current location), matching characteristic vector is calculated to it, which is input in the disaggregated model, It is sub-divided into a second level classification, belongs to the second level classification from historical operating data library further according to the matching characteristic vector The highest sample work of similarity under the measurement of weighting cosine similarity or weighted euclidean distance is matched in operation sample For template samples；The synthetic operation sequence that data transformation synthesizes the start-stop point demand for meeting simulated operation is carried out to the template samples Column are accurately simulated based on mouse action analogue technique and execute the synthetic operation sequence.
It is preferred according to the present invention, the step (1), the operation segmentation, it is assumed that certain section of original mouse sequence of operation is by N A mouse action point is constituted, and N is uncertain integer, and one section of sequence may be 100 points, it is also possible to 150 points, or 1234 points are entirely to be determined by the data of actual acquisition；Include:
A, original mouse sequence of operation cutting is M according to the definition of atom operation by operation segmentation item1A atom operation；The member Operation includes single-click operation, idler wheel scroll operation, moving operation, drag operation；The single-click operation include left button single-click operation, Right button single-click operation, middle key single-click operation, the single-click operation of other function keys；Left button single-click operation refers to that left button presses coordinate bit Set the operation behavior that the offset for bouncing coordinate position with left button is less than n pixel distance；Right button single-click operation refers to right button It presses coordinate position and right button bounces operation behavior of the offset less than n pixel distance of coordinate position；Middle key etc. other Function key refers to the alternate function key that certain mouse devices are additionally arranged, these function keys and left mouse button, right button is same has list Operation is hit, but there may be other function.The single-click operation of other function keys is analogous to left button single-click operation, right button clicks behaviour Make, different function key can be distinguished and be treated.Such as: the single-click operation of function key A and function key B are considered as two class atom operations；Idler wheel rolling Dynamic operation refers to the operation that an idler wheel rolls；Moving operation refers to since static, starting has significantly accelerated process, to static Terminate, terminate before have the operation of obvious moderating process；Generally there is apparent mouse quiescent time between two sections of mobile atom operations, separately Outside, settable acceleration rate threshold, it is desirable that the initial acceleration of one section of independent mobile atom operation is greater than threshold value, ending negative acceleration It, accordingly can be very long by one section less than threshold value (threshold value reasonably specific value and screen resolution, mouse the factors such as are arranged related) Mobile messaging sequence be cut into degree of precision by the mobile multistage independence moving operation generated of the actual multiple mouse of user. Drag operation moves the operation bounced after relatively long distance after referring to certain mouse button down；It is unsatisfactory for the keystroke behaviour of offset requirement Make, that is, is unsatisfactory for the keystroke operation of offset requirement.
B, according to the definition of combination operation by M1A atom operation is combined into M2A combination operation；M1, M2 are and real data feelings The related uncertain integer of condition.Then the operating process of this section of mouse action session contains M by this2The sequence of a combination operation is retouched It states；The combination operation include double click operation, movement and single-click operation, movement and double click operation, it is other do not constitute double click operation, The idler wheel continuous rolling operation combination of multiple scroll operations (a string or), mobile and single-click operation, the member of movement and double click operation Operation；I.e. can not group be combined into double-click, movement and that clicks click atom operation.M2A combination operation is category；The category Several user's history operation datas under catalogue are known as operation sample of the user under the category；
C, data cleansing filters off abnormal data caused by due to accidentally touching etc. sampling error or user, abnormal data It include: the operation sample of the time interval value exception there are neighboring message, there are the mutations of the coordinate position of neighboring message Sample.
There are the operation samples of the time interval value exception of adjacent mouse action point for example: a left button single-click operation In, timestamp that left button is pressed is 1500 milliseconds, the timestamp that left button bounces is 3000 milliseconds, and normal condition servant clicks Operating time is spaced in a few tens of milliseconds between several hundred milliseconds, this may be caused by system Caton or the few situation of user The lower result that can be generated.It is related that timestamp is mutated the factors such as reasonable value and mouse information frequency, the equipment performance of threshold value；In the presence of The sample that the coordinate position of neighboring message mutates is for example: in one section of movement, k-th point of coordinate is (150,200), kth+ The coordinate of 1 point sports (200,300), this is usually that system Caton, user accidentally touch etc. caused by reasons.It is mutated the conjunction of threshold value It is related to manage the factors such as value and screen resolution, mouse setting.
According to the present invention preferably, the purpose of step (2), extraction cluster feature is for different user, by level-one class Other operation sample is further subdivided into several second level classifications according to its practical feature having respectively, divide second level classification be for Subsequent template matching process is matched to cluster feature template as similar as possible under the conditions of same operation, category mould as far as possible Type is related to user.The reason is that the operation behavior feature of different user has different distributions in feature space, and grasp The input parameter for making synthesis module only describes the basic demand for being intended to simulated operation, is only capable of calculating a small amount of operation behavior spy Sign further clusters the matching mesh that can make subsequent match process to existing complete operation using more fully operation behavior feature Mark is limited in a cluster, to reduce matching error under the background of mouse track identification model, ensure behavioural characteristic similitude The success rate detected with simulated operation by various actions.
The cluster feature includes:
1) for single-click operation, the coordinate including key pressing and the time interval, key pressing that bounce, i.e., (x, y, dt)；(x, Y) refer to the coordinate of key pressing；The time interval that dt refers to key pressing and bounces；
2) for double click operation, cluster feature, first time single-click operation including single-click operation twice terminate to second The time difference that single-click operation starts；Total 2 screen coordinates and 3 time intervals, it may be assumed that (x1, y1, dt1, dt2, x2, y2, dt3)；
3) idler wheel continuous rolling is operated, time interval including adjacent roll mouse operating point adjacent scrolls up The time interval of mouse action point, the adjacent time interval for scrolling down through mouse action point, scrolling up to be switched to scrolls down through When time interval, scroll down through the statistical nature group for being switched to the essential characteristic of time interval when scrolling up, the idler wheel The statistical nature group at message time interval refers to the value that each essential characteristic repeatedly obtains in continuous idler wheel scroll operation Maximum value, minimum value, mean value, standard deviation, the point quantity, variance, median, the degree of bias, kurtosis of sequence；Such as: M times idler wheel rolls The idler wheel continuous rolling operation of composition can at least calculate the M-1 adjacent time interval rolled twice, to this M-1 time Statistical natures such as maximum value, minimum value, mean value that interval calculation goes out, as " adjacent rolling time interval " this essential characteristic Statistical nature group.M is the positive integer greater than 1.
4) for moving operation, the operating point quantity including constituting the moving operation, total time-consuming, what initial point to terminal was constituted Deflection, the line segment length of vector (object vector), curve of approximation (broken line) total length, average movement speed, mobile efficiency (being equal to line segment length divided by curve of approximation total length), the statistical nature group per adjacent distance between two points, when per adjacent point-to-point transmission Between the statistical nature group that is spaced, the statistical nature group of movement speed sequence accelerates the statistical nature group of degree series, acceleration Statistical nature group, the statistical nature group of deviation angle, the statistical nature group of curvature, the statistical nature group of angular speed, local motion effect Rate sequence (be equal to per in adjacent 3 points A, B, C, AC long divided by AB long and BC's long and) statistical nature group, local offset sequence The statistical nature group of column (be equal to per in adjacent 3 points A, B, C, projector distance of the point B to straight line AC), total drift amount sequence is (to removing Each point outside whole story point, total drift amount be equal to its to straight line determined by object vector projector distance) statistical nature Group, statistical nature group and maximum speed point, most greatly of the total drift than sequence (be equal to total drift amount divided by curve total length) Speed point, minimum acceleration point, maximum angular rate point, maximum curvature point, maximum offset point, minimum local motion efficient point exist Space percent position and percentage of time position in operation trace；
5) for mobile and single-click operation, movement and double click operation, including single-click operation, double click operation, moving operation pair The cluster feature answered further includes the pause time between adjacent combination operation, i.e. moving operation, the pause between single-click operation Pause time between time difference and moving operation, double click operation；
6) for drag operation: further including key pressing to when starting mobile including the corresponding cluster feature of moving operation Between interval, terminate be moved to the time interval that key bounces.
It is general by it for the step of being not necessarily to moving operation, omitting cluster and classification, directly training transversal normal distribution model Rate distribution generates time interval parameter；
Frame designed by the present invention be to the selection of cluster feature it is expansible, can both add new feature to strengthen Model can also adapt to the demand of different application with selected characteristic subset.
Preferred according to the present invention, the step (2) extracts matching characteristic, comprising:
A, input is intended to the parameter of simulated operation, and the parameter for being intended to simulated operation refers to: for mobile and single-click operation, referring to shifting The target position (terminal) moved and clicked；For drag operation, refer to the beginning and end of dragging；Wherein starting point can be omitted defeated Enter, voluntarily obtains in stepb；Under by taking mobile and single-click operation simulation as an example, drag operation is similar；
B, the current coordinate position of mouse is voluntarily obtained, the starting point of the movement to be simulated and single-click operation is obtained.
C, the matching characteristic for being intended to simulated operation is calculated, matching characteristic is input to trained disaggregated model, finds and is intended to The most like second level classification of simulated operation；The matching characteristic includes: start-stop point distance, start-stop point vector direction angle, starting point seat Mark, stop coordinate；
D, it by the method for measuring similarity such as weighting cosine similarity metric or weighted euclidean distance measurement, finds most like Template samples.
Preferred according to the present invention, the step (3), operation synthesizes and simulation, comprising:
For giving trained transversal normal distribution model without mobile operation, generates and be intended to needed for simulated operation Time interval constructs the mouse action point sequence of simulated operation according to the time interval parameter of generation；For example, generating left button list Hit operation left button press, bounce between time interval T after, can be with the movement point sequence of constructing analog left button single-click operation [(X, Y, 0, left button are pressed), (X, Y, T, left button bounce)]；It the use of transversal normal distribution is in order to avoid normal distribution small probability Generate the possibility of unusual feature.
For the mobile operation of needs, it is based on trained disaggregated model, similitude matching strategy and historical operating data, Template samples of the most like historical operation sample as motion track are obtained, a series of transformation are carried out to the template samples Operation: include:
1. carrying out coordinate translation to motion track template, keep its starting point consistent with the track starting point of simulated operation is intended to；
2. carrying out coordinate rotation around starting point to motion track template, making the vector direction of its start-stop point determination and being intended to simulate behaviour The direction of work is consistent；
3. carrying out the constant track of starting point to motion track template to scale, the line segment length for determining its start-stop point, which is equal to, to be intended to The start-stop point distance of simulated operation；The start-stop point of the track and the track for being intended to simulated operation has been completely coincident at this time；
4. coordinate position, the timestamp to motion track template add disturbance, certain randomness is made it have；At this time should Motion track template is made for describing one section of complete mouse action message sequence with the initial point of the mouse action message sequence For reference time zero point, processing, the i.e. execution for the mouse action to be simulated are formatted to the track.Typical case is such as The software analog form or other hardware simulation modes of " message insertion " under Windows system.When simulating message, relative time Stamp is explicitly entered not as parameter, and as the reference time of Message Simulation point, it is implicit to embody, it may be assumed that when certain message is specified Between point simulate the message.
5. simulating the mouse action message sequence, the method is as follows:
(I) message sequence (x of one section of desire simulated operation is inputted1, y1, 0, a1), (x2, y2, t2, a2) ..., (xN, yN, tN, aN)；(xi, yi) it is the screen position coordinate that mouse action occurs, tiTime for mouse action time of origin relative to starting point Interval, aiFor mouse action type (such as: left button is pressed, and left button bounces, and mouse is mobile etc.).1≤I≤N；
(II) record start time T0；
(III) first message (x is simulated1, y1, a1), record executes time T1, and the simulation for calculating second message waits Time DT=t2+T0-T1；
(IV) after waiting the DT time, second message is simulated, record executes time T2, calculate the DT=t of third message3+ T0-T2；
(V) and so on, the simulated time of i-th of message is referring to initial time T0For zero point, tiFor relative time delay, ginseng Examine the time of present system time calculating currently also needed to wait for；
(VI) when n-th Message Simulation is completed, mouse action message sequence simulation is completed.
A kind of synthesis and simulation system of the behavior of user's mouse, including sequentially connected data acquisition and procession module, mould Type training module, operation synthesis and analog module；
The data acquisition and procession module is used for: acquiring the original mouse data of user, original mouse data in real time Refer to the original mouse sequence of operation that (x, y, t, action) four-tuple, that is, mouse action point or mouse information point are constituted, (x, y) For the coordinate of mouse on the screen, unit is pixel；T is the timestamp that mouse action generates, and unit is millisecond or microsecond； Action is the type of mouse action, including mouse is mobile, idler wheel scrolls up, left button is pressed or bounce, right button by It is lower or bounce, middle key pressing or bounce and other function key pressing or bounce；It will be converted after original mouse data manipulation segmentation Specifically to operate, specific operation includes that left button is clicked, right button is clicked, left double click, middle key are clicked, other function key list Right button is clicked after left double click, movement after left button is clicked, moved after hitting, moving, idler wheel continuous rolling, left button pull, right button drags It drags, then after filtering out via data cleansing and the abnormal data caused by reasons such as accidentally touching as system call or user, deposit history behaviour Make database；
The model training module is used for: for without mobile operation, without mobile operation include left button click, it is right Key is clicked, left double click, middle key are clicked, other function key is clicked, idler wheel continuous rolling, and training transversal normal distribution model is pressed Its probability distribution generates time interval parameter, and time interval parameter is the time interval between each mouse information；For needing The operation to be moved, need mobile operation include it is mobile after left button click, move after left double click, it is mobile after right button click, Left button pulls, right button pulls, and extracts cluster feature and matching characteristic to the data in historical operating data library, constructs training set； Cluster feature clusters each category, for training Clustering Model using cluster result as historical operation sample Second level class label；Matching characteristic combination Clustering Model is used to contain movement to each to the second level class label of sample labeling One disaggregated model of category operation training of operation, for not knowing the behaviour of affiliated second level classification under the category still Make sample, which is classified as a specific second level under the category according to the matching characteristic vector of the sample Classification, to match the higher template samples of similarity (under be referred to as template samples)；Category is above-mentioned through system Pre-process clicked the left button that sorts out, left double click, it is mobile after left button is clicked, left button pulls, second level classification is calculated by cluster Method voluntarily clusters the classification of generation under each specific category according to data sample；
The operation synthesis is used for analog module: for the operation without movement, using giving trained truncation just The message sequence that time interval needed for state distributed model generates simulated operation, constructing analog operate；For needing mobile behaviour Make, inputs mobile starting point coordinate and terminal point coordinate (wherein starting point can be defaulted using mouse current location), it is calculated The matching characteristic vector is input in the disaggregated model by matching characteristic vector out, is sub-divided into a second level classification, then One is matched from the operation sample for belonging to the second level classification in historical operating data library remaining according to the matching characteristic vector The highest sample of similarity is as template samples under the measurement of string similarity or Euclidean distance；Data change is carried out to the template samples The synthetic operation sequence for synthesizing the start-stop point demand for meeting simulated operation is changed, is accurately simulated and is held based on mouse action analogue technique The row synthetic operation sequence.
The invention has the benefit that
1, a kind of method for simulating real user mouse action behavior of the present invention can be grasped from a small amount of user's mouse Make to extract common mouse action in track, and synthesizes and simulate new mouse action behavior, these new mouse action rows For be to a certain extent have user's mouse behavioural characteristic.The present invention is first and is designed based on data and user characteristics Mouse Behavior modeling invention.
2, the generated data that generates of the present invention and simulated operation can be used for various models based on user's mouse behavior and be The performance test of system and reinforcing, the automation based on mankind's mouse behavior associated verification code crack etc..To mobile and left button The Preliminary Simulation of single-click operation tests display, and success rate has increase trend with the historical operating data amount used.
Detailed description of the invention
Fig. 1 is synthesis and the analogy method flow diagram of user's mouse behavior of the present invention；
Fig. 2 is the flow diagram of model training stage of the present invention；
Fig. 3 is the flow diagram of operation of the present invention synthesis and dummy run phase；
Specific embodiment
The present invention is further qualified with embodiment with reference to the accompanying drawings of the specification, but not limited to this.
Embodiment 1
A kind of synthesis and analogy method of the behavior of user's mouse, as shown in Figure 1, comprising:
(1) data acquisition and procession
When user is using PC or by browser access Web site, adopted based on the Hook PC end data realized Collection program or the Web end data capture program based on JavaScript can acquire the original mouse data of user in real time, former Beginning mouse data refers to the original mouse operation that (x, y, t, action) four-tuple, that is, mouse action point or mouse information point are constituted Sequence, (x, y) are the coordinate of mouse on the screen, and unit is pixel；T is the timestamp that mouse action generates, and unit is milli Second or microsecond；Action is the type of mouse action, including mouse is mobile, idler wheel scrolls up, left button is pressed or bullet It rises, right button is pressed or bounced, middle key pressing or is bounced and other function key pressing or is bounced；
Specific operation will be converted into after original mouse data manipulation segmentation, specific operation include left button click, right button Click, left double click after left button is clicked, moved after left double click, middle key are clicked, other function key is clicked, moved, it is mobile after it is right Key is clicked, idler wheel continuous rolling, left button pull, right button pulls, then is filtered out via data cleansing by system call or user's mistake After the abnormal data caused by reasons such as touching, it is stored in historical operating data library；
(2) model training
As shown in Fig. 2, including that left button is clicked, right button is clicked, left button without mobile operation for without mobile operation Double-click, middle key is clicked, other function key is clicked, idler wheel continuous rolling, training transversal normal distribution model, raw by its probability distribution At time interval parameter, time interval parameter is the time interval between each mouse information, e.g., single-click operation: key mapping is pressed Lower message, key mapping bounce the time interval between message；
For the mobile operation of needs, left double click, shifting after needing mobile operation to click, move including left button after movement Right button is clicked after dynamic, left button pulls, right button pulls, and extracts cluster feature to the data in historical operating data library and matching is special Sign constructs training set；Cluster feature clusters each category for training Clustering Model, using cluster result as The second level class label of historical operation sample；Matching characteristic combination Clustering Model is used for the second level class label of sample labeling To one disaggregated model of each category operation training containing moving operation, for not knowing institute still under the category Belong to the operation sample of second level classification, which is classified as under the category according to the matching characteristic vector of the sample A specific second level classification, to match the higher template samples of similarity (under be referred to as template samples)；Category Be it is above-mentioned clicked through system pretreatment come the left button that sorts out, left double click, it is mobile after left button is clicked, left button pulls, second level Classification is voluntarily to cluster the classification of generation according to data sample under each specific category by clustering algorithm；Such as: to movement The category sample that left button is clicked afterwards is clustered, and generates several sample clusters, left button clicks one after the as movement of each cluster A second level classification (such as: single-click operation after the first kind is mobile, single-click operation, etc. after the second class is mobile) under grade classification, this A little clusters are automatically generated by Clustering Model algorithm according to cluster feature, do not need engineer's sub-clustering rule.
The model for one generic operation (calling category in the following text) being clustered, being subdivided into second level classification, with the model to such Sample under operation carries out classification (calling second level classification in the following text) label in class；Matching characteristic is to be released in cluster feature by simulation demand Character subset, for be intended to simulated operation requirement under match a second level classification, then by the second level classification sample set use In the specific historical data target (i.e. template samples) of matching；
(3) operation synthesis and simulation
As shown in figure 3, a behavior based on mouse action can be decomposed into the combination of several category mouse actions, Successively simulating each category mouse action can be realized the simulation of mouse behavior.
For being generated needed for simulated operation using trained transversal normal distribution model is given without mobile operation The message sequence that time interval, constructing analog operate；
For needing mobile operation, (wherein starting point can be defaulted and make for the mobile starting point coordinate of input and terminal point coordinate With mouse current location), matching characteristic vector is calculated to it, which is input in disaggregated model, by it It is sub-divided into a second level classification, belongs to the operation of the second level classification from historical operating data library further according to the matching characteristic vector Matched in sample one weighting cosine similarity or weighted euclidean distance measurement under the highest sample of similarity as mould Plate sample；The synthetic operation sequence that data transformation synthesizes the start-stop point demand for meeting simulated operation is carried out to the template samples, It is accurately simulated based on mouse action analogue technique and executes the synthetic operation sequence.
Embodiment 2
According to a kind of synthesis and analogy method of the behavior of user's mouse described in embodiment 1, difference is:
Step (1), operation segmentation, it is assumed that certain section of original mouse sequence of operation is made of N number of mouse action point, comprising:
A, original mouse sequence of operation cutting is M according to the definition of atom operation by operation segmentation item1A atom operation；Atom operation Including single-click operation, idler wheel scroll operation, moving operation, drag operation；Single-click operation includes that left button single-click operation, right button are clicked Operation, middle key single-click operation, the single-click operation of other function keys；Left button single-click operation refers to that left button presses coordinate position and left button The offset for bouncing coordinate position is less than the operation behavior of n pixel distance；Right button single-click operation refers to that right button presses coordinate The offset that position and right button bounce coordinate position is less than the operation behavior of n pixel distance；The other functions key such as middle key refers to The alternate function key that certain mouse devices are additionally arranged, these function keys and left mouse button, right button be same to have single-click operation, but There may be other function.The single-click operation of other function keys is analogous to left button single-click operation, right button single-click operation, different function Key can be distinguished and be treated.Such as: the single-click operation of function key A and function key B are considered as two class atom operations；Idler wheel scroll operation refers to one The operation that secondary idler wheel rolls；Moving operation refers to since static, starting has significantly accelerated process, to before static end, terminating There is the operation of obvious moderating process；Generally there is apparent mouse quiescent time between two sections of mobile atom operations, in addition, settable add Threshold speed, it is desirable that the initial acceleration of one section of independent mobile atom operation is greater than threshold value, ending negative acceleration is less than threshold value (threshold It is related to be worth the factors such as reasonable specific value and screen resolution, mouse setting), it accordingly can be by one section very long of mobile messaging sequence Column are cut into degree of precision by the mobile multistage independence moving operation generated of the actual multiple mouse of user.Drag operation refers to The operation bounced after relatively long distance is moved after certain mouse button down；It is unsatisfactory for the keystroke operation of offset requirement, that is, is unsatisfactory for The keystroke operation that offset requires.
B, according to the definition of combination operation by M1A atom operation is combined into M2A combination operation；Then this section of mouse action session Operating process by this contain M2The sequence description of a combination operation；Combination operation includes double click operation, idler wheel continuous rolling behaviour Make, mobile and single-click operation, movement and double click operation, other do not constitute double click operation, movement and single-click operation, movement and double-click The atom operation of operation；I.e. can not group be combined into double-click, movement and that clicks click atom operation.M2A combination operation is level-one class Not；Several user's history operation datas under category catalogue are known as operation sample of the user under the category；
C, data cleansing filters off abnormal data caused by due to accidentally touching etc. sampling error or user, abnormal data It include: the operation sample of the time interval value exception there are neighboring message, there are the mutations of the coordinate position of neighboring message Sample.
There are the operation samples of the time interval value exception of adjacent mouse action point for example: a left button single-click operation In, timestamp that left button is pressed is 1500 milliseconds, the timestamp that left button bounces is 3000 milliseconds, and normal condition servant clicks Operating time is spaced in a few tens of milliseconds between several hundred milliseconds, this may be caused by system Caton or the few situation of user The lower result that can be generated.It is related that timestamp is mutated the factors such as reasonable value and mouse information frequency, the equipment performance of threshold value；In the presence of The sample that the coordinate position of neighboring message mutates is for example: in one section of movement, k-th point of coordinate is (150,200), kth+ The coordinate of 1 point sports (200,300), this is usually that system Caton, user accidentally touch etc. caused by reasons.It is mutated the conjunction of threshold value It is related to manage the factors such as value and screen resolution, mouse setting)
Embodiment 3
According to a kind of synthesis and analogy method of the behavior of user's mouse described in embodiment 1, difference is:
The purpose of step (2), extraction cluster feature is for different user, by the operation sample of category respectively according to it The feature actually having is further subdivided into several second level classifications, and dividing second level classification is in order to which subsequent template matching process is most It is flux matched to the template as similar as possible of cluster feature under the conditions of same operation, category model is related to user.Reason It is that the operation behavior feature of different user has different distributions in feature space, and operates the input parameter of synthesis module only The basic demand for being intended to simulated operation is described, is only capable of calculating a small amount of operation behavior feature, to existing complete operation using more Comprehensive operation behavior feature, which further clusters, can make the matching tar-get of subsequent match process in a cluster, in mouse It marks and reduces matching error under the background of track identification model, ensures that behavioural characteristic similitude and simulated operation are examined by various actions The success rate of survey.
Cluster feature includes:
1) for single-click operation, the coordinate including key pressing and the time interval, key pressing that bounce, i.e., (x, y, dt)；(x, Y) refer to the coordinate of key pressing；The time interval that dt refers to key pressing and bounces；
2) for double click operation, cluster feature, first time single-click operation including single-click operation twice terminate to second The time difference that single-click operation starts；Total 2 screen coordinates and 3 time intervals, it may be assumed that (x1, y1, dt1, dt2, x2, y2, dt3)；
3) idler wheel continuous rolling is operated, time interval including adjacent roll mouse operating point adjacent scrolls up The time interval of mouse action point, the adjacent time interval for scrolling down through mouse action point, scrolling up to be switched to scrolls down through When time interval, scroll down through the statistical nature group for being switched to the essential characteristic of time interval when scrolling up, idler wheel message The statistical nature group of time interval refers to the sequence for the value that each essential characteristic repeatedly obtains in continuous idler wheel scroll operation Maximum value, minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis；Such as: M times idler wheel rolls composition Idler wheel continuous rolling operation can at least calculate the M-1 adjacent time interval rolled twice, to this M-1 time interval The statistical natures such as calculated maximum value, minimum value, mean value, the as statistics of " adjacent rolling time interval " this essential characteristic Feature group.M is the positive integer greater than 1.
4) for moving operation, the operating point quantity including constituting the moving operation, total time-consuming, what initial point to terminal was constituted Deflection, the line segment length of vector (object vector), curve of approximation (broken line) total length, average movement speed, mobile efficiency (being equal to line segment length divided by curve of approximation total length), the statistical nature group per adjacent distance between two points, when per adjacent point-to-point transmission Between the statistical nature group that is spaced, the statistical nature group of movement speed sequence accelerates the statistical nature group of degree series, acceleration Statistical nature group, the statistical nature group of deviation angle, the statistical nature group of curvature, the statistical nature group of angular speed, local motion effect Rate sequence (be equal to per in adjacent 3 points A, B, C, AC long divided by AB long and BC's long and) statistical nature group, local offset sequence The statistical nature group of column (be equal to per in adjacent 3 points A, B, C, projector distance of the point B to straight line AC), total drift amount sequence is (to removing Each point outside whole story point, total drift amount be equal to its to straight line determined by object vector projector distance) statistical nature Group, statistical nature group and maximum speed point, most greatly of the total drift than sequence (be equal to total drift amount divided by curve total length) Speed point, minimum acceleration point, maximum angular rate point, maximum curvature point, maximum offset point, minimum local motion efficient point exist Space percent position and percentage of time position in operation trace；
It is made of for example, setting one section of movement five shift action points, the description of this five points are as follows: point P1(100,100,0, Mouse is mobile), point P2(110,105,10, mouse is mobile), point P3(130,110,20, mouse is mobile), point P4(160,115,30, Mouse is mobile), point P5(200,120,40, mouse is mobile).If i-th of point PiBe described as (xi, yi, ti, ai), then:
The operating point quantity for constituting the moving operation is N=5, and the mobile total time-consuming of the section is T=40 milliseconds.
The deflection of object vector is vector's Radian number θ, in this exampleRadian, as δ x=0, θ is if δ y > 0Arc Degree, θ is if δ y > 0Radian.
The line segment length of object vector is
Curve of approximation total length is are as follows: Wherein The distance of i+1 point is arrived for i-th point.
Average movement speed is are as follows:
Moving efficiency is are as follows:
Distance per adjacent point-to-point transmission is δ si, δ s in this example1=11.18, δ s2=20.62, δ s3=30.41, δ s4= 40.31.Its statistical nature group is maximum value, minimum value, mean value, the standard of sequence (11.18,20.62,30.41,40.31) Difference, point quantity, variance, median, the degree of bias, kurtosis.For example, maximum value δ smax=max (11.18,20.62,30.41,40.31) =40.31, mean valueStandard deviationPartially Degree
Time interval sequence per adjacent point-to-point transmission is δ ti=ti+1-ti, i=1 ..., N-1, δ tiIt is arrived for i-th point The time interval of i+1 point, δ t in this example1=δ t2=δ t3=δ t4=10.Its statistical nature group be sequence (10,10, 10,10) maximum value, minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis.
Movement speed sequence is are as follows:In this examplev2= 2.062 v3=3.041, v4=4.031；Its statistical nature group is the maximum of sequence (1.118,2.062,3.041,4.031) Value, minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis.
Translational acceleration sequence is are as follows:Wherein δ vi=vi+1-viIt is arrived for i-th of speed The changing value of i+1 speed.In this example (note: this example is because of δ siOnly remain into after decimal point 2, final rounding error cause it is even plus The data of speed movement calculate the feature of non-even acceleration, can be more accurate in actual computer operation, which will It ignores)；Its statistical nature group is maximum value, minimum value, mean value, the standard of sequence (0.0944,0.0979,0.099) Difference, point quantity, variance, median, the degree of bias, kurtosis.
Deviation angle sequence is are as follows: δ αi=αi- θ, i=1 ..., N-1, wherein θ is the object vector direction of the motion track Angle,It is vectorDeflection.In this example Its statistical nature group be sequence (0.2663,0.0476, -0.0322, - 0.0730) maximum value, minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis.
Curvature sequence is are as follows:Wherein δ θi=αi+1-αiFor 3 points of determinations adjacent in motion track Two adjacent motion-vectors angle change.In this example Its statistical nature group is the maximum value, most of sequence (- 0.01956, -0.00387, -0.00134) Small value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis.
Angular speed sequence is are as follows:In this example Its statistical nature group be sequence (- 0.02187, -0.00798, - 0.00408) maximum value, minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis.
Local motion efficiency sequence is are as follows:WhereinFor i-th point of distance to the i-th+2 point, (that is: adjacent 3 points true The chord length of fixed arc).In this example Its statistical nature group be the maximum value of sequence (0.9946,0.9992,0.9998), minimum value, mean value, standard deviation, point quantity, Variance, median, the degree of bias, kurtosis.
Local offset refers to the vertical line section of the second o'clock straight line determined to first and third point in per adjacent three points Length, sequence are are as follows:Wherein AI-1, i+1、BI-1, i+1、 CI-1, i+1For the parameter for the linear equation Ax+By+C=0 that (i-1)-th point and i+1 point determine, coordinate can be brought into solution of equation ?.In this example Its statistical nature group is maximum value, minimum value, mean value, standard deviation, the point quantity, side of sequence (1.581,0.981,0.707) Difference, median, the degree of bias, kurtosis.
Total drift amount refer to each point in addition to the start and the end points only to beginning and end determine straight line vertical line section length Degree, sequence are are as follows:Wherein A0、B0、C0It is determined for beginning and end straight The parameter of line equation Ax+By+C=0 can bring coordinate into solution of equation and obtain.In this exampleIts statistical nature Group be the maximum value of sequence (2.942,3.922,2.942), minimum value, mean value, standard deviation, point quantity, variance, median, The degree of bias, kurtosis.
Ratio of the total drift than being equal to each point total drift amount and curve total length, sequence are are as follows: In this exampleIts statistical nature group be sequence (0.0287, 0.0383,0.0287) maximum value, minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis.
Maximum speed point, peak acceleration point, minimum acceleration point, maximum angular rate point, maximum offset point, minimum part Space percent position Ls and percentage of time position Lt of the mobile efficient point in operation trace, by taking maximum speed point as an example, He is similar: in this examplev2=2.062, v3=3.041, v4=4.031, wherein maximum speed vmax=v4It can determine Its adopted value is in point P4, point P5Midpoint, it may be assumed that Its time percent position is it relative to the ratio between the timestamp of starting point and total time-consuming, i.e.,It will point P4.5Project to line segment P1FNIt, can be according to plane geometry on determining straight line L Calculate subpoint (intersection point) Its space percent position is line SectionLength and object vector long D ratio, i.e.,
5) for mobile and single-click operation, movement and double click operation, including single-click operation, double click operation, moving operation pair The cluster feature answered further includes the pause time between adjacent combination operation, i.e. moving operation, the pause between single-click operation Pause time between time difference and moving operation, double click operation；
6) for drag operation: further including key pressing to when starting mobile including the corresponding cluster feature of moving operation Between interval, terminate be moved to the time interval that key bounces.
It is general by it for the step of being not necessarily to moving operation, omitting cluster and classification, directly training transversal normal distribution model Rate distribution generates time interval parameter；
Frame designed by the present invention be to the selection of cluster feature it is expansible, can both add new feature to strengthen Model can also adapt to the demand of different application with selected characteristic subset.
Step (2) extracts matching characteristic, comprising:
A, input is intended to the parameter of simulated operation, and the parameter for being intended to simulated operation refers to: for mobile and single-click operation, referring to shifting The target position (terminal) moved and clicked；For drag operation, refer to the beginning and end of dragging；Wherein starting point can be omitted defeated Enter, voluntarily obtains in stepb；Under by taking mobile and single-click operation simulation as an example, drag operation is similar；
B, the current coordinate position of mouse is voluntarily obtained, the starting point of the movement to be simulated and single-click operation is obtained.
C, the matching characteristic for being intended to simulated operation is calculated, matching characteristic is input to trained disaggregated model, finds and is intended to The most like second level classification of simulated operation；Matching characteristic includes: start-stop point distance, start-stop point vector direction angle, starting point coordinate, stops Point coordinate；
D, it by the method for measuring similarity such as weighting cosine similarity metric or weighted euclidean distance measurement, finds most like Template samples.
Embodiment 4
According to a kind of synthesis and analogy method of the behavior of user's mouse described in embodiment 1, difference is:
Step (3), operation synthesis and simulation, comprising:
For giving trained transversal normal distribution model without mobile operation, generates and be intended to needed for simulated operation Time interval constructs the mouse action point sequence of simulated operation according to the time interval parameter of generation；For example, generating left button list Hit operation left button press, bounce between time interval T after, can be with the movement point sequence of constructing analog left button single-click operation [(X, Y, 0, left button are pressed), (X, Y, T, left button bounce)]；It the use of transversal normal distribution is in order to avoid normal distribution small probability Generate the possibility of unusual feature.
For the mobile operation of needs, it is based on trained disaggregated model, similitude matching strategy and historical operating data, Template samples of the most like historical operation sample as motion track are obtained, a series of transformation are carried out to the template samples Operation: include:
1. carrying out coordinate translation to motion track template, keep its starting point consistent with the track starting point of simulated operation is intended to；
2. carrying out coordinate rotation around starting point to motion track template, making the vector direction of its start-stop point determination and being intended to simulate behaviour The direction of work is consistent；
3. carrying out the constant track of starting point to motion track template to scale, the line segment length for determining its start-stop point, which is equal to, to be intended to The start-stop point distance of simulated operation；The start-stop point of the track and the track for being intended to simulated operation has been completely coincident at this time；
4. coordinate position, the timestamp to motion track template add disturbance, certain randomness is made it have；At this time should Motion track template is made for describing one section of complete mouse action message sequence with the initial point of the mouse action message sequence For reference time zero point, processing, the i.e. execution for the mouse action to be simulated are formatted to the track.Typical case is such as The software analog form or other hardware simulation modes of " message insertion " under Windows system.When simulating message, relative time Stamp is explicitly entered not as parameter, and as the reference time of Message Simulation point, it is implicit to embody, it may be assumed that when certain message is specified Between point simulate the message.
5. simulating mouse action message sequence, the method is as follows:
(I) message sequence (x of one section of desire simulated operation is inputted1, y1, 0, a1), (x2, y2, t2, a2) ..., (xN, yN, tN, aN)；(xi, yi) it is the screen position coordinate that mouse action occurs, tiTime for mouse action time of origin relative to starting point Interval, aiFor mouse action type (such as: left button is pressed, and left button bounces, and mouse is mobile etc.).1≤I≤N；
(II) record start time T0；
(III) first message (x is simulated1, y1, a1), record executes time T1, and the simulation for calculating second message waits Time DT=t2+T0-T1；
(IV) after waiting the DT time, second message is simulated, record executes time T2, calculate the DT=t of third message3+ T0-T2；
(V) and so on, the simulated time of i-th of message is referring to initial time T0For zero point, tiFor relative time delay, ginseng Examine the time of present system time calculating currently also needed to wait for；
(VI) when n-th Message Simulation is completed, mouse action message sequence simulation is completed.
Embodiment 5
A kind of synthesis and analogy method of the behavior of user's mouse, comprising:
(1) the original mouse track data for acquiring user X, therefrom extracts and effectively clicks, double-clicks, moving, moving simultaneously Click, move and double-click, pull, wheel operation it is several, this be level-one class of operation, if wherein left double-click operation sample has 100, left button drag operation sample has 50；
(2) feature is extracted to each level-one class of operation respectively；
(3) model foundation.For the operation without movement, such as: left double-click operation, directly to this 100 left double clicks The construction feature for operating sample carries out normal distribution fitting, and converts transversal normal distribution mould for each normal distribution model Type.Specific features include: first time left button press with left button bounce between time interval Dt1, first time left button bounces and the Time interval Dt2 that secondary left button is pressed, second of left button press the time interval Dt3 bounced with second of left button.For containing There is mobile operation, such as: left button drag operation is based on its cluster feature, this 50 left buttons are pulled behaviour using DBSCAN algorithm If being polymerized to Ganlei (set and be polymerized to K class) as sample, as the second level class label of this 50 sample, it is then based on this 50 samples Matching characteristic and its corresponding label, using GBDT algorithm one disaggregated model of training, which can be by the one of input A matching characteristic vector is categorized into one of K second level classification；
(4) for case require " left button pull certain target to designated position, the then left double click target " behavior, vacation If mouse current location and target present bit are equipped with certain distance, then it can will decompose the behavior are as follows: be moved to target position, pull Target is double-clicked to designated position, original place, that is, is moved, pulled, double-clicking three level-one operations, operation room is randomly provided within the scope of certain Time delay, such as 1000 milliseconds, 500 milliseconds.The usually extraneous given input of the target location coordinate that operation is related to, It can be identified by handle or HTML element positions or image-recognizing method obtains coordinates of targets.The movement that decomposites, left button are dragged It drags, left double-click operation, successively executes (5) respectively and arrive (7), it should be noted that after the completion of moving operation simulation, mouse is currently sat Cursor position will be updated；
(5) to movement and the following operation of drag operation execution containing moving operation: by taking left button drag operation as an example, by inputting Coordinate of ground point parameter and mouse current location parameter, calculate the matching characteristic of the operation, inputted as feature vector left Key drag operation corresponding GBDT disaggregated model obtains (wherein, the GBDT of second level classification belonging to the drag operation to be simulated Have the right to randomly choose in the highest Ganlei of class probability if output can be, the highest class of probability can also be directly selected), if There are 5 samples under the second level classification, then it again can be from this 5 second level classes very by weighting cosine similarity, weighted euclidean distance One operation template of matching in this (wherein, matching the mode of an operation template, to can be similarity highest, be also possible to Machine selection), into (6).
(6) for the left double-click operation without movement, according to corresponding to tri- features of Dt1, Dt2, Dt3 in (3) Three transversal normal distributions can generate one group (Dt1, Dt2, Dt3) at random respectively, this left double-click operation parameter synthesized, close At left button single-click operation sequence can be described as: [(x, y, 0, left-down), (x, y, Dt1, left-up), (x, y, Dt1+ Dt2,left-down),(x,y,Dt1+Dt2+Dt3,left-up)]；For containing mobile operation, such as (5) left button pulls behaviour Make, the progress of template sequence obtained by (5) coordinate translation, coordinate rotation, coordinate scaling, disturbance are added, a starting point can be obtained and exist Mouse current location, terminal are in the track sets for clicking target position, which can be described as: [(x1,y1,0,left-down), (x2,y2,t2,move),……,(xn,yn,tn,mouse-move),(xn-1,yn-1,tn-1,move),(xn,yn,tn,left- up)]；
(7) standardization of chronomere is made to the sequence of operation obtained by (6), and simulates execution.
After the completion of when the movement, left button dragging, left double click decomposited in (4), these three operate simulation execution, case Mouse behavior i.e. simulate complete.
In the experiment of designated user's fidelity of simulation, the present invention acquires the mouse action data of 24 users, using this The mouse behavior synthesis of invention and simulation system are simulated experiment.Using classical in mouse behavior field of identity authentication Based on authentication framework (distinguishing whether sample belongs to some user) that is multiple mobile and clicking behavior, verified as benchmark model Effect is simulated, mobile and in the case where click behavior as an identification sample using 20 times, the benchmark model is to true mouse The average misclassification rate of behavior is lower than 2%, and to the flat of the mouse behavior simulated the present invention is based on the truthful data of 24 users Equal misclassification rate is in 60% or more, misclassification rate median 85% or more.I.e. to most users, mouse behavior energy that the present invention simulates Enough more realistically modelling customer behavior features, with 85% or more the detection for passing through reference identity authentication model at percent of pass.It can , it is expected that the present invention (distinguishes sample and belongs to real user or script robot) to existing robot behavior detection model There to be higher percent of pass.
The mouse behavior that existing mouse event analogy method simulates does not have user behavior characteristics, to authentication mould The percent of pass of type is lower than 2%, and robot behavior detection model is also easy to be identified.
Embodiment 6
A kind of synthesis and simulation system of the behavior of user's mouse, including sequentially connected data acquisition and procession module, mould Type training module, operation synthesis and analog module；
Data acquisition and procession module is used for: acquiring the original mouse data of user in real time, original mouse data refer to The original mouse sequence of operation that (x, y, t, action) four-tuple, that is, mouse action point or mouse information point are constituted, (x, y) are mouse The coordinate of mark on the screen, unit is pixel；T is the timestamp that mouse action generates, and unit is millisecond or microsecond；action For the type of mouse action, including mouse is mobile, idler wheel scrolls up, left button is pressed or bounces, right button is pressed or bullet It rises, middle key pressing or bounce and other function key pressing or bounce；It is specific by being converted into after original mouse data manipulation segmentation Operation, specific operation includes that left button is clicked, right button is clicked, left double click, middle key are clicked, other function key is clicked, moved Right button is clicked after left double click, movement after left button is clicked, moved afterwards, idler wheel continuous rolling, left button pull, right button is pulled, then passed through It is filtered out as data cleansing after the abnormal data caused by reasons such as accidentally touching as system call or user, is stored in historical operating data Library；
Model training module is used for: for without mobile operation, without mobile operation include left button click, right button list Hit, left double click, middle key are clicked, other function key is clicked, idler wheel continuous rolling, training transversal normal distribution model, by its generally Rate distribution generates time interval parameter, and time interval parameter is the time interval between each mouse information；For needing to move Dynamic operation, need mobile operation include it is mobile after left button click, move after left double click, it is mobile after right button click, left button It pulls, right button dragging, cluster feature and matching characteristic is extracted to the data in historical operating data library, construct training set；Cluster Feature clusters each category, for training Clustering Model using cluster result as the second level of historical operation sample Class label；Matching characteristic combination Clustering Model is used to contain moving operation to each to the second level class label of sample labeling One disaggregated model of category operation training, for the operation sample of second level classification belonging to not knowing still under the category This, the specific second level class which is classified as under the category according to the matching characteristic vector of the sample Not, to match the higher template samples of similarity (under be referred to as template samples)；Category is above-mentioned pre- through system Handle clicked the left button that sorts out, left double click, it is mobile after left button is clicked, left button pulls, second level classification is by clustering algorithm The classification of generation is voluntarily clustered according to data sample under each specific category；
Operation synthesis is used for analog module: for being divided using trained truncated normal is given without mobile operation The message sequence that time interval needed for cloth model generates simulated operation, constructing analog operate；It is defeated for needing mobile operation Enter mobile starting point coordinate and terminal point coordinate (wherein starting point can be defaulted using mouse current location), matching is calculated to it The matching characteristic vector is input in disaggregated model by feature vector, a second level classification is sub-divided into, further according to the matching Feature vector matched from the operation sample for belonging to the second level classification in historical operating data library one in cosine similarity or The highest sample of similarity is as template samples under the measurement of Euclidean distance；Data transformation is carried out to the template samples and synthesizes symbol The synthetic operation sequence for closing the start-stop point demand of simulated operation is accurately simulated based on mouse action analogue technique and executes synthesis behaviour Make sequence.

Claims (6)

1. a kind of synthesis and analogy method of the behavior of user's mouse characterized by comprising
(1) data acquisition and procession
The original mouse data of user are acquired in real time, and original mouse data refer to that (x, y, t, action) four-tuple, that is, mouse is dynamic Make the original mouse sequence of operation that point or mouse information point are constituted, (x, y) is the coordinate of mouse on the screen, and unit is pixel Point；T is the timestamp that mouse action generates, and unit is millisecond or microsecond；Action is the type of mouse action, including mouse moves It is dynamic, idler wheel scrolls up, left button is pressed or bounces, right button is pressed or is bounced, middle key pressing or is bounced and other function Energy key pressing is bounced；
Specific operation will be converted into after original mouse data manipulation segmentation, specific operation include left button is clicked, right button is clicked, Right button list after left double click, movement after left button is clicked, moved after left double click, middle key are clicked, other function key is clicked, moved Hit, idler wheel continuous rolling, left button pull, right button pulls, then after filtering out abnormal data via data cleansing, be stored in historical operation Database；
(2) model training
For without mobile operation, without mobile operation include left button is clicked, right button is clicked, left double click, middle key are clicked, Other function key clicks, idler wheel continuous rolling, and training transversal normal distribution model generates time interval by its probability distribution and joins Number, time interval parameter is the time interval between each mouse information；
For the mobile operation of needs, after needing mobile operation to click, move including left button after movement after left double click, movement Right button is clicked, left button pulls, right button pulls, and extracts cluster feature and matching characteristic, structure to the data in historical operating data library Build training set；Cluster feature clusters each category, grasps cluster result as history for training Clustering Model Make the second level class label of sample；Matching characteristic combination Clustering Model is used for each the second level class label of sample labeling One disaggregated model of category operation training containing moving operation, for not knowing affiliated second level still under the category The operation sample of classification, the disaggregated model are classified as one under the category according to the matching characteristic vector of the sample Specific second level classification, to match the higher template samples of similarity；Category is to sort out through system pretreatment Left button click, left double click, it is mobile after left button is clicked, left button pulls, second level classification is by clustering algorithm in each specific level-one The classification of generation is voluntarily clustered under classification according to data sample；
(3) operation synthesis and simulation
For using the time needed for giving trained transversal normal distribution model generation simulated operation without mobile operation The message sequence that interval, constructing analog operate；
For the mobile operation of needs, mobile starting point coordinate and terminal point coordinate are inputted, matching characteristic vector is calculated to it, it will The matching characteristic vector is input in the disaggregated model, is sub-divided into a second level classification, further according to the matching characteristic to Amount matches one in weighting cosine similarity or is added from the operation sample for belonging to the second level classification in historical operating data library The highest sample of similarity is as template samples under the measurement of power Euclidean distance；Data transformation is carried out to the template samples to synthesize Meet the synthetic operation sequence of the start-stop point demand of simulated operation, which is executed based on the simulation of mouse action analogue technique Sequence.
2. the synthesis and analogy method of a kind of user's mouse behavior according to claim 1, which is characterized in that step (1), The operation segmentation, it is assumed that certain section of original mouse sequence of operation is made of N number of mouse action point, comprising:
A, original mouse sequence of operation cutting is M according to the definition of atom operation by operation segmentation item1A atom operation；The atom operation Including single-click operation, idler wheel scroll operation, moving operation, drag operation；The single-click operation includes left button single-click operation, right button Single-click operation, middle key single-click operation, the single-click operation of other function keys；Left button single-click operation refer to left button press coordinate position with The offset that left button bounces coordinate position is less than the operation behavior of n pixel distance；Right button single-click operation refers to that right button is pressed The offset that coordinate position and right button bounce coordinate position is less than the operation behavior of n pixel distance；Idler wheel scroll operation is Refer to the operation that an idler wheel rolls；Moving operation refers to since static, starting has significantly accelerated process, to static end, knot The operation of the obvious moderating process of Shu Qianyou；Drag operation moves the behaviour bounced after relatively long distance after referring to certain mouse button down Make；
B, according to the definition of combination operation by M1A atom operation is combined into M2A combination operation；The combination operation includes double-clicking behaviour Make, the operation of idler wheel continuous rolling, mobile and single-click operation, movement and double click operation, other constitute double click operation, movement and single Hit the atom operation of operation, movement and double click operation；M2A combination operation is category；Several users under category catalogue Historical operating data is known as operation sample of the user under the category；
C, data cleansing filters off abnormal data, and abnormal data includes: the operation of the time interval value exception there are neighboring message Sample, there are the samples that the coordinate position of neighboring message mutates.
3. the synthesis and analogy method of a kind of user's mouse behavior according to claim 1, which is characterized in that step (2), The cluster feature includes:
1) for single-click operation, the coordinate including key pressing and the time interval, key pressing that bounce, i.e., (x, y, dt)；(x, y) is The coordinate that digital is pressed；The time interval that dt refers to key pressing and bounces；
2) for double click operation, cluster feature, first time single-click operation including single-click operation twice terminate to clicking for the second time Operate the time difference started；
3) idler wheel continuous rolling is operated, time interval, the adjacent mouse that scrolls up including adjacent roll mouse operating point The time interval of operating point, the adjacent time interval for scrolling down through mouse action point, scrolling up is switched to when scrolling down through Time interval scrolls down through the statistical nature group for being switched to the essential characteristic of time interval when scrolling up, the essential characteristic Statistical nature group refer to value that each essential characteristic repeatedly obtains in continuous idler wheel scroll operation sequence maximum value, Minimum value, mean value, standard deviation, point quantity, variance, median, the degree of bias, kurtosis；
4) for moving operation, the operating point quantity including constituting the moving operation, total time-consuming, the vector that initial point to terminal is constituted Deflection, line segment length, curve of approximation total length, average movement speed, mobile efficiency, the statistics per adjacent distance between two points Feature group, the statistical nature group per adjacent point-to-point transmission time interval, the statistical nature group of movement speed sequence accelerate degree series Statistical nature group, the statistical nature group of acceleration, the statistical nature group of deviation angle, the statistical nature group of curvature, angular speed Statistical nature group, the statistical nature group of local motion efficiency sequence, the statistical nature group of local offset sequence, total drift amount sequence The statistical nature group of column, total drift is than statistical nature group and the maximum speed point of sequence, peak acceleration point, minimum acceleration Spend the sky of point, maximum angular rate point, maximum curvature point, maximum offset point, minimum local motion efficient point in operation trace Between percent position and percentage of time position；
5) corresponding for mobile and single-click operation, movement and double click operation, including single-click operation, double click operation, moving operation Cluster feature further includes the pause time between adjacent combination operation, i.e. moving operation, the dead time between single-click operation Difference and the pause time between moving operation, double click operation；
6) for drag operation: further including key pressing to starting between the mobile time including the corresponding cluster feature of moving operation Every, terminate to be moved to the time interval that key bounces.
4. the synthesis and analogy method of a kind of user's mouse behavior according to claim 1, which is characterized in that step (2), Extract matching characteristic, comprising:
A, input is intended to the parameter of simulated operation, and the parameter for being intended to simulated operation refers to: for mobile and single-click operation, referring to movement simultaneously The target position clicked；For drag operation, refer to the beginning and end of dragging；
B, the current coordinate position of mouse is voluntarily obtained, the starting point of the movement to be simulated and single-click operation is obtained；
C, the matching characteristic for being intended to simulated operation is calculated, matching characteristic is input to trained disaggregated model, finds and be intended to simulate Operate most like second level classification；The matching characteristic includes: start-stop point distance, start-stop point vector direction angle, starting point coordinate, stops Point coordinate；
D, by weighting cosine similarity metric or weighted euclidean distance measured similarity measure, most like template is found Sample.
5. the synthesis and analogy method of a kind of user's mouse behavior according to claim 1 to 4, which is characterized in that step Suddenly (3), operation synthesis and simulation, comprising:
For giving trained transversal normal distribution model without mobile operation, the time needed for being intended to simulated operation is generated Interval, the mouse action point sequence of simulated operation is constructed according to the time interval parameter of generation；
For needing mobile operation, it is based on trained disaggregated model, similitude matching strategy and historical operating data, is obtained Template samples of one most like historical operation sample as motion track carry out a series of transformation to the template samples and grasp Make: including:
1. carrying out coordinate translation to motion track template, keep its starting point consistent with the track starting point of simulated operation is intended to；
2. carrying out coordinate rotation around starting point to motion track template, the vector direction for determining its start-stop point and desire simulated operation Direction is consistent；
3. carrying out the constant track of starting point to motion track template to scale, the line segment length for determining its start-stop point, which is equal to, to be intended to simulate The start-stop point distance of operation；
4. coordinate position, the timestamp to motion track template add disturbance, the motion track template is for describing one section at this time Complete mouse action message sequence, using the initial point of the mouse action message sequence as reference time zero point, to the track It is formatted processing, the i.e. execution for the mouse action to be simulated；
5. simulating the mouse action message sequence, the method is as follows:
(I) message sequence (x of one section of desire simulated operation is inputted1,y1,0,a1),(x2,y2,t2,a2),…,(xN,yN,tN,aN)； (xi,yi) it is the screen position coordinate that mouse action occurs, tiIt is mouse action time of origin relative between the time of starting point Every aiFor mouse action type, 1≤i≤N；
(II) record start time T0；
(III) first message (x is simulated1,y1,a1), record executes time T1, calculates the simulation waiting time DT of second message =t2+T0–T1；
(IV) after waiting the DT time, second message is simulated, record executes time T2, calculate the DT=t of third message3+T0– T2；
(V) and so on, the simulated time of i-th of message is referring to initial time T0For zero point, tiFor relative time delay, with reference to working as The time currently also needed to wait for that preceding system time calculates；
(VI) when n-th Message Simulation is completed, mouse action message sequence simulation is completed.
6. a kind of synthesis and simulation system of the behavior of user's mouse, including sequentially connected data acquisition and procession module, model Training module, operation synthesis and analog module；
The data acquisition and procession module is used for: acquiring the original mouse data of user in real time, original mouse data refer to The original mouse sequence of operation that (x, y, t, action) four-tuple, that is, mouse action point or mouse information point are constituted, (x, y) are mouse The coordinate of mark on the screen, unit is pixel；T is the timestamp that mouse action generates, and unit is millisecond or microsecond；action For the type of mouse action, including mouse is mobile, idler wheel scrolls up, left button is pressed or bounces, right button is pressed or bullet It rises, middle key pressing or bounce and other function key pressing or bounce；It is specific by being converted into after original mouse data manipulation segmentation Operation, specific operation includes that left button is clicked, right button is clicked, left double click, middle key are clicked, other function key is clicked, moved Right button is clicked after left double click, movement after left button is clicked, moved afterwards, idler wheel continuous rolling, left button pull, right button is pulled, then passed through After filtering out abnormal data by data cleansing, it is stored in historical operating data library；
The model training module is used for: for without mobile operation, without mobile operation include left button click, right button list Hit, left double click, middle key are clicked, other function key is clicked, idler wheel continuous rolling, training transversal normal distribution model, by its generally Rate distribution generates time interval parameter, and time interval parameter is the time interval between each mouse information；For needing to move Dynamic operation, need mobile operation include it is mobile after left button click, move after left double click, it is mobile after right button click, left button It pulls, right button dragging, cluster feature and matching characteristic is extracted to the data in historical operating data library, construct training set；Cluster Feature clusters each category, for training Clustering Model using cluster result as the second level of historical operation sample Class label；Matching characteristic combination Clustering Model is used to contain moving operation to each to the second level class label of sample labeling One disaggregated model of category operation training, for the operation sample of second level classification belonging to not knowing still under the category This, the specific second level class which is classified as under the category according to the matching characteristic vector of the sample Not, to match the higher template samples of similarity；Category be the left button sorted out through system pretreatment click, Left double click, it is mobile after left button is clicked, left button pulls, second level classification be by clustering algorithm under each specific category according to number The classification of generation is voluntarily clustered according to sample；
The operation synthesis is used for analog module: for being divided using trained truncated normal is given without mobile operation The message sequence that time interval needed for cloth model generates simulated operation, constructing analog operate；It is defeated for needing mobile operation Enter mobile starting point coordinate and terminal point coordinate, matching characteristic vector is calculated to it, which is input to described In disaggregated model, it is sub-divided into a second level classification, is belonged to from historical operating data library further according to the matching characteristic vector The highest sample of similarity under the measurement of cosine similarity or Euclidean distance is matched in the operation sample of the second level classification This is as template samples；The synthesis behaviour that data transformation synthesizes the start-stop point demand for meeting simulated operation is carried out to the template samples Make sequence, which is executed based on the simulation of mouse action analogue technique.
CN201711148377.1A 2017-11-17 2017-11-17 A kind of synthesis of user's mouse behavior and analogy method and system Active CN107908300B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711148377.1A CN107908300B (en) 2017-11-17 2017-11-17 A kind of synthesis of user's mouse behavior and analogy method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711148377.1A CN107908300B (en) 2017-11-17 2017-11-17 A kind of synthesis of user's mouse behavior and analogy method and system

Publications (2)

Publication Number Publication Date
CN107908300A CN107908300A (en) 2018-04-13
CN107908300B true CN107908300B (en) 2019-08-13

Family

ID=61846232

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711148377.1A Active CN107908300B (en) 2017-11-17 2017-11-17 A kind of synthesis of user's mouse behavior and analogy method and system

Country Status (1)

Country Link
CN (1) CN107908300B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109189519B (en) * 2018-07-26 2021-10-19 哈尔滨工业大学（威海） Universal user desktop behavior simulation system and method
CN109102061A (en) * 2018-08-10 2018-12-28 哈尔滨工业大学（威海） A kind of group of subscribers behavior generation system meeting human dynamics rule, equipment and storage medium
CN109902474B (en) * 2019-03-01 2020-11-03 北京奇艺世纪科技有限公司 Method and device for determining moving track of moving object in sliding verification code
CN110135487A (en) * 2019-05-09 2019-08-16 国网山东省电力公司滨州供电公司 A kind of computer user mouse Behavior modeling method
CN110377354B (en) * 2019-06-04 2020-09-01 哈尔滨工业大学（威海） Automatic generation method of software operation sequence
CN110532874B (en) * 2019-07-23 2022-11-11 深圳大学 Object attribute recognition model generation method, storage medium and electronic device
CN112070132A (en) * 2020-08-25 2020-12-11 北京百度网讯科技有限公司 Sample data construction method, device, equipment and medium
CN112306363B (en) * 2020-10-30 2022-04-29 腾讯科技（深圳）有限公司 Mouse simulation method and device, display equipment and storage medium
CN112967418B (en) * 2021-01-29 2023-01-06 惠州市大圣和乘科技有限公司 Vehicle-mounted data record display method and device, computer equipment and storage medium
CN115202495B (en) * 2022-09-08 2022-12-16 深圳市湘凡科技有限公司 Mouse hardware simulation movement method, device, equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101963851A (en) * 2009-07-25 2011-02-02 英华达(上海)电子有限公司 Mobile equipment and implementation method, receiving host and implementation method as well as communication system
CN102830930A (en) * 2012-08-15 2012-12-19 Tcl集团股份有限公司 Treatment method and device for keyboard keys and multimedia terminal
CN103699822A (en) * 2013-12-31 2014-04-02 同济大学 Application system and detection method for users' abnormal behaviors in e-commerce based on mouse behaviors

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080154573A1 (en) * 2006-10-02 2008-06-26 Microsoft Corporation Simulating new input devices using old input devices

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101963851A (en) * 2009-07-25 2011-02-02 英华达(上海)电子有限公司 Mobile equipment and implementation method, receiving host and implementation method as well as communication system
CN102830930A (en) * 2012-08-15 2012-12-19 Tcl集团股份有限公司 Treatment method and device for keyboard keys and multimedia terminal
CN103699822A (en) * 2013-12-31 2014-04-02 同济大学 Application system and detection method for users' abnormal behaviors in e-commerce based on mouse behaviors

Also Published As

Publication number Publication date
CN107908300A (en) 2018-04-13

Similar Documents

Publication Publication Date Title
CN107908300B (en) A kind of synthesis of user's mouse behavior and analogy method and system
CN109446927B (en) Double-person interaction behavior identification method based on priori knowledge
US6072494A (en) Method and apparatus for real-time gesture recognition
CN110135487A (en) A kind of computer user mouse Behavior modeling method
CN103530540B (en) User identity attribute detection method based on man-machine interaction behavior characteristics
Chambers et al. Hierarchical recognition of intentional human gestures for sports video annotation
CN109189519B (en) Universal user desktop behavior simulation system and method
CN108182728A (en) A kind of online body-sensing three-dimensional modeling method and system based on Leap Motion
Hua et al. Collaborative active visual recognition from crowds: A distributed ensemble approach
CN105426929B (en) Object shapes alignment device, object handles devices and methods therefor
Fujii Data-driven analysis for understanding team sports behaviors
CN109241709A (en) User behavior recognition method and device based on the verifying of sliding block identifying code
CN111210111B (en) Urban environment assessment method and system based on online learning and crowdsourcing data analysis
CN112183643B (en) Hard rock tension-shear fracture identification method and device based on acoustic emission
CN109903053B (en) Anti-fraud method for behavior recognition based on sensor data
CN108596148A (en) A kind of construction worker's labour state analysis system based on computer vision and method
CN106022251A (en) Abnormal double-person interaction behavior recognition method based on vision co-occurrence matrix sequence
CN109784015A (en) A kind of authentication identifying method and device
CN110379522A (en) A kind of disease popularity trend predicting system and method
CN106778714A (en) LDA face identification methods based on nonlinear characteristic and model combination
CN113077679A (en) Interactive virtual assembly teaching method and system
Abdulaal et al. Real-time synchronization in neural networks for multivariate time series anomaly detection
CN106990768A (en) MKPCA batch process fault monitoring methods based on Limited DTW
CN116881017B (en) Collaborative virtual maintenance training system and method
CN110347579B (en) Deep learning test case selection method based on neuron output behavior pattern

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Wang Bailing

Inventor after: Hu Shujie

Inventor after: Bai Jun

Inventor after: Huang Junheng

Inventor after: Wang Wei

Inventor after: Xin Guodong

Inventor before: Hu Shujie

Inventor before: Bai Jun

Inventor before: Wang Bailing

Inventor before: Huang Junheng

Inventor before: Wang Wei

Inventor before: Xin Guodong