CN107426148A - A kind of anti-reptile method and system based on running environment feature recognition - Google Patents
A kind of anti-reptile method and system based on running environment feature recognition Download PDFInfo
- Publication number
- CN107426148A CN107426148A CN201710203203.4A CN201710203203A CN107426148A CN 107426148 A CN107426148 A CN 107426148A CN 201710203203 A CN201710203203 A CN 201710203203A CN 107426148 A CN107426148 A CN 107426148A
- Authority
- CN
- China
- Prior art keywords
- client
- code
- server
- selection
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The present invention discloses a kind of anti-reptile method and system based on running environment feature recognition, and the present invention relates to anti-crawler technology field, solves crawlers identification and the anti-technical problem for climbing strategy implement.It is main to include producing new strategy bag and the option code for the operation of new strategy bag, update server current strategies bag using new strategy bag and build the feature classification white list on new strategy bag operation characteristic data；Option code is sent to client, response is then sent and asks to client；By client, according to option code, selectively operation reserve bag carries out server response, obtain corresponding selection code characteristic and backout feature data to server；Characteristic corresponding to analysis option code and option code, calculates the feature classification of client and judges whether to belong to feature classification white list, the client implementation access control to being not belonging to feature classification white list.
Description
Technical field
The present invention relates to reptile identification and anti-crawler technology field, and in particular to a kind of based on running environment feature recognition
Anti- reptile method and system.
Background technology
With the arrival in big data epoch, data become more and more important.Valuable data are analyzed from data, it is first
First need substantial amounts of data.Data on internet receive the pass of enterprises and individuals due to its publicity, magnanimity, popularity
Note.Many web crawlers are exploited, and gather the various data on internet.But web crawlers can bring it is many negative
Influence.Web crawler meeting short time a large amount of request servers, cause server performance to decline；Some reptiles can malice collection
A large amount of public datas, collect selling, property infringement.Also the data of number of site have very high value density, or
Enterprise is not intended to oneself disclosed information and gathered easily by web crawlers, all implements different anti-reptile measures, as identifying code,
Conversational check, access frequency etc. distinguish web crawlers or the truly artificially access to system.But web crawlers is more
Kind is various, also takes many technological means for breaking through anti-reptiles.Identifying code is such as identified by machine recognition, artificial stamp,
By splicing access request etc. around conversational check and using the simulation multi-user access such as Address Proxy.By being run to reptile
The feature recognition of environment, web crawlers can be effectively identified, prevents leaking data.
The content of the invention
For above-mentioned prior art, present invention aims at provide a kind of anti-reptile side based on running environment feature recognition
Method and system, solution prior art reptile embodiment party, which constantly accesses server and largely obtains information, causes server operation speed
Degree is slow and also existence information resource is obtained by batch and the technical problem such as steals.
To reach above-mentioned purpose, the technical solution adopted by the present invention is as follows：
A kind of anti-reptile method, comprises the following steps：
Step 1, new strategy bag and the option code for the operation of new strategy bag are produced, worked as using new strategy bag renewal server
Preceding strategy, which wraps, simultaneously builds the feature classification white list on new strategy bag operation characteristic data, can be by separate server or
Home server carries out generation operation；
Step 2, option code is sent to client, then send response and ask to client, can pass through stand-alone service
Device or home server are transmitted or received operation；
Step 3, by client, according to option code, selectively operation reserve bag carries out server response, obtains corresponding selection code
Characteristic and backout feature data to server；
Characteristic corresponding to step 4, analysis option code and option code, calculates the feature classification of client and judges
Whether feature classification white list is belonged to, the client implementation access control to being not belonging to feature classification white list.
In the above method, described step 1, the cycle produces new strategy bag and the option code for the operation of new strategy bag.
In the above method, described step 5, comprise the following steps：
Step 4.1, by memory module receive characteristic in predetermined time interval；
Step 4.2, characteristic of the memory module simultaneously in analysis time section is accessed by server processing module, calculated
Go out the feature classification of client and judge whether to belong to feature classification white list；
Step 4.2.1, it is legal the client for belonging to feature classification white list to be marked, then jumps to step 1；
Step 4.2.2, to being not belonging to the client implementation access control of feature classification white list.
A kind of anti-reptile method based on running environment feature recognition, comprises the following steps：
Step 1, the server end cycle produce with different run time program functions as element Jacobian matrix and
For selection of mapping character strings different elements into Jacobian matrix, and the white list of running environment feature classification is set, then
The current function matrix of server is updated by Jacobian matrix, random character string alternatively code is generated in server end, then
Send option code and response is asked to client；
Step 2, in client selection is gone out by option code decision-making, then show that selection is corresponding in Jacobian matrix
The run time of element, then the run time of option code and corresponding element is sent to server；
Step 3, in server end option code and run time are analyzed, calculate client running environment feature classification, sentence
Break and to be not belonging in white list the client of running environment feature classification and to the client implementation access control policy.
In the above method, described step 1, in addition to cycle produce the selection subcharacter for including selection Ziwen eigen
Code.
In the above method, described step 3, comprise the following steps：
Step 3.1, inquiry server currently select subcharacter code, obtain it is currently used in Jacobian matrix and for analyzing
With the selection subvalue for calculating initial setting up；
Step 3.2, select current function matrix all option codes and institute from client in preset time section
The run time of corresponding element, client running environment feature class is calculated by means clustering algorithm or machine learning algorithm
Not；
Step 3.3, judge to be not belonging to the client of running environment feature classification in white list and the client is marked
Illegal and implementation access control policy.
In the above method, described step 3, received by the memory module of server in preset time section and come from client
Option code and corresponding element run time.
A kind of anti-crawler system based on running environment feature recognition, including
Server, including characteristics algorithm module, data interface module, memory module, data analysis module and access process
Module, characteristics algorithm module export the strategy bag for updating and performing to server in itself；
Client, the option code generated by characteristics algorithm module is received by data interface module；
Described client is by option code implementation strategy bag；
Described server, the spy by client executing strategy bag response is received by data interface module corresponding selection code
Data are levied, the characteristic of corresponding selection code is also exported to memory module, data analysis module and calculates and deposit by data interface module
Store up in module the characteristic of corresponding selection code and by result of calculation feedback information to access processing module, access processing module by
Feedback information is to the client executing predetermined policy.
In such scheme, described strategy bag, including the function square with different run time program functions as element
Battle array, for mapping character strings, into Jacobian matrix, the selection of different elements is sub and includes the selection subcharacter of selection Ziwen eigen
Code.
A kind of server based on running environment feature recognition, with anti-reptile function, including
Characteristics algorithm module, the cycle produce with different run time program functions as element Jacobian matrix, be used for
Mapping character strings selection of different elements into Jacobian matrix is sub and includes the selection subcharacter code of selection Ziwen eigen；
Data interface module, for client interaction data, export as caused by characteristics algorithm module option code to visitor
The run time of family end and reception from client corresponding selection code；
Memory module, receive the corresponding selection code run time received in data interface module scheduled time section
Data；
Data analysis module, calculate in memory module data in scheduled time section and client is judged by result of calculation
Running environment feature classification；
Access processing module, the client running environment feature classification calculated by data analysis module is to client executing
Predetermined policy.
Compared with prior art, beneficial effects of the present invention：
The present invention is collected by randomly choosing code so as to which Stochastic Decision-making goes out in Jacobian matrix to be used for the program function of computing
Its run time in certain time section, judges whether it belongs to valid operation, dramatically increases after classifying according to cluster feature
Find the possibility of reptile client and with increasing for data is collected, can significantly reduce white list False Rate；
The generation that subcharacter code is selected in the present invention is random and Jacobian matrix is variable function matrix so that client
Need the Jacobian matrix computing that performs all different every time, so as to substantially increase the difficulty that reptile is cracked.
Brief description of the drawings
Fig. 1 is the server architecture block diagram of the embodiment of the present invention；
Fig. 2 is the anti-reptile verification method implementing procedure figure of the embodiment of the present invention.
Embodiment
All features disclosed in this specification, or disclosed all methods or during the step of, except mutually exclusive
Feature and/or step beyond, can combine in any way.
Running environment refers to that client accesses the environment of the local application operation of service end service；Such as access Web
The running environment of the browser of the page, access the wechat software runtime environment of the wechat page.
White list refers to that a kind of differentiate through overwriting is feature set of records ends in normal client running environment；Such as can
The operation on different operating system, different hardware platforms is carried out with the browser accessed Web page and carries out actual test, collection
Run time inside patent describing mode, after carrying out machine learning, pattern-recognition, neural network learning, carry out algorithmically
Feature is sorted out, the set for having classification of formation.
Option code can be when client accesses, and carry out randomly generating character string, current time such as be obtained, plus random
Integer, and make hash algorithm generation.
Selection is one section of program, the solution that specific algorithm can be solved with equation with many unknowns, then solution sorts from small to large, entered
After row carries out modulus to the columns of Jacobian matrix, the operation function operation of this row is taken per a line successively；To sub this section of journey of selection
Sequence, after the Hash for carrying out text, filename corresponding to cryptographic Hash and this program is recorded in database.
Each value of Jacobian matrix is the index of one section of operation program, according to Jacobian matrix obtained value, can be found
One operation program；This operation program can be one section of JavaScr ipt program in Web page, when running it, need
The time of this section of program code of operation is recorded, then by being analyzed operating time log to judge client-side program
Running environment.
The present invention will be further described below in conjunction with the accompanying drawings：
A kind of anti-reptile method and system of running environment feature recognition, are specifically included：
S1, server cycle produce Jacobian matrix, selection, select subcharacter code, update server program, set and access
Running environment feature classification white list；
S2, server generation option code, option code is concurrently sent to client, requesting client response；
S3, client select function and operation in Jacobian matrix according to the option code received, by selection, then
Option code, each function operation time are sent to server；
S4, server store option code, each function operation time to data memory module；
S5, server are analyzed option code, the data of each function operation time of storage, calculate active client
The feature classification of running environment；
S6, the strategy according to configuration, access control policy is implemented to the access not in white list；
In the above method, Jacobian matrix, option code, the key step bag of selection caused by server feature algoritic module
Include：
S11, Jacobian matrix form N*N matrixes by the program function of different execution times, are identified as f (i, j)；
S12, selection are the programs by character string maps to several f (i, j)；
S13, selection subcharacter code are the text features for selecting subprogram；
S14, renewal server program, replace Jacobian matrix, selection of legacy version, and record current selection subcharacter
Code；
S15, running environment white list are the client running environment of default Lawful access；
In the above method, server generation option code, concurrently send option code to client, requesting client response it is main
Step includes：
The character string that S21, option code are randomly generated；
It is server settings that whether S22, server, which ask client's response,；
In the above method, server is analyzed option code, the data of each function operation time of storage, is calculated and is worked as
The feature classification key step of preceding client running environment includes：
S51, the selection subcharacter code for inquiring about server current record, judge currently used Jacobian matrix and selection；
S52, selection current function matrix corresponding to certain time section data storage, calculate active client operation ring
The feature classification in border, different recognizers can be used, such as cluster, machine learning；
S53, mark do not meet running environment feature classification white list and obtain client access；
In the above method, characteristics algorithm generation module, data acquisition module data interface module, data storage are specifically included
Module, data analysis module, access processing module；Characteristics algorithm module mainly produces Jacobian matrix, selection, selection subcharacter
Code, option code；Data acquisition module data interface module mainly sends and receives data；Data memory module is mainly used in depositing
Store up option code, the value of Jacobian matrix under different characteristic setting that client returns；Data analysis module is mainly to data storage
Analyzed, calculate running environment feature classification；Access processing module is mainly the strategy according to setting, implements access control.
Embodiment 1
Wherein, as shown in figure 1, the server includes a data acquisition module data interface module 1, a characteristics algorithm generates
Module 2, a data memory module 3, a data analysis module 4 and an access processing module 5.
Each functional module possessed function is described below：
Data acquisition module data interface module 1 is used to send and receive data.
Characteristics algorithm generation module 2 is used for generating function matrix, selection subcharacter code and selection.
Data memory module 3 is used to store the option code under different characteristic setting of client return, Jacobian matrix fortune
Row time data.
Data analysis module 4 is used to analyze the data in data memory module, and comes from client receiving
Jacobian matrix run time data when, with the data calculate running environment feature classification, whether received with authentication server
Access from reptile.
Access processing module 5 is used for the strategy according to setting, to client implementation access control.
As shown in Fig. 2 present embodiments providing a kind of anti-reptile verification method, this method is real using above-mentioned anti-crawler system
Existing, it comprises the following steps：
Step 101, server generating function matrix f (i, j), selection subcharacter code and selection and server program is updated
And running environment feature classification white list；
Step 102, server generates option code, concurrently send option code to client, requesting client response；
Step 103, client receives option code, and the function in Jacobian matrix is selected by selection；
Step 104, the function that client operation is selected, and gather function operation time data；
Step 105, client sends option code and run time data to server；
Step 106, option code and run time data of the server storage from client；
Step 107, server is analyzed the option code of storage, function operation time data, calculates existing customer
Hold the feature classification of running environment；
Step 108, server enters step 109 according to the strategy of configuration when the client is in running environment white list,
When the client is not to enter step 110 in running environment white list；
Step 109, server allows client to access, and terminates flow；
Step 110, server forbids client to access, and terminates flow.
In order that those skilled in the art are better understood from technical scheme, come for a specific example below
Explanation：
Jacobian matrix f (3,3) is stored with setting server, i.e., has 9 functions (function f (1,1), f in the matrix
(1,2), f (1,3), (2,1), f (2,2), f (2,3), f (3,1), f (3,2), f (3,3).
Server generates option code S at random, and option code S is sent to client, waits client response.
After client receives option code S, parsed, and function f (1,1), f are chosen by selection according to analysis result
(2,1), f (2,1), (3,1).
Client is separately operable function f (1,1), f (2,1), f (2,1), (3,1) and generating run time T1, T2, T3,
T4.In the case of server end requesting client response, client sends out run time data T1, T2, T3, T4 and option code
Deliver to server.
After server receives data T1, T2, T3, T4 and option code from client, after being stored, trigger data
Analysis module is analyzed the data.
Specifically, server can be by time data T1, T2, T3, T4 and the historical data in data memory module are carried out pair
Than to calculate the feature classification M1 of active client running environment.
After this feature classification is obtained, data analysis module sends this feature classification M1 to access control module.
After access control module obtains feature classification M1, running environment white list is accessed, if it in white list, takes
The client of business device running continues to access server, if it not in white list, server forbids the client to continue to visit
Ask server.
In above-mentioned verification process, option code is randomly generated by server, and further, operation function is by selection
Random selection, this can greatly increase reptile and crack difficulty.
The foregoing is only a specific embodiment of the invention, but protection scope of the present invention is not limited thereto, any
Belong to those skilled in the art the invention discloses technical scope in, the change or replacement that can readily occur in, all should
It is included within the scope of the present invention.
Claims (10)
- A kind of 1. anti-reptile method, it is characterised in that comprise the following steps：Step 1, new strategy bag and the option code for the operation of new strategy bag are produced, update the current plan of server using new strategy bag Slightly wrap and build the feature classification white list on new strategy bag operation characteristic data；Step 2, option code is sent to client, then send response and ask to client；Step 3, by client, according to option code, selectively operation reserve bag carries out server response, obtains the spy of corresponding selection code Data and backout feature data are levied to server；Characteristic corresponding to step 4, analysis option code and option code, calculates the feature classification of client and judges whether Belong to feature classification white list, the client implementation access control to being not belonging to feature classification white list.
- 2. a kind of anti-reptile method according to claim 1, it is characterised in that described step 1, cycle produce new strategy Bag and the option code for the operation of new strategy bag.
- 3. a kind of anti-reptile method according to claim 1, it is characterised in that described step 4, comprise the following steps：Step 4.1, by memory module receive characteristic in predetermined time interval；Step 4.2, characteristic of the memory module simultaneously in analysis time section is accessed by server processing module, calculate visitor The feature classification at family end simultaneously judges whether to belong to feature classification white list；Step 4.2.1, it is legal the client for belonging to feature classification white list to be marked, then jumps to step 1；Step 4.2.2, to being not belonging to the client implementation access control of feature classification white list.
- A kind of 4. anti-reptile method based on running environment feature recognition, it is characterised in that comprise the following steps：Step 1, there are different run time program functions as the Jacobian matrix of element in the generation of server end cycle and be used for Selection of mapping character strings different elements into Jacobian matrix, the program documentaion of selection is then calculated by hash algorithm Condition code alternatively subcharacter code, and the white list of running environment feature classification is set, then pass through Jacobian matrix more new demand servicing The current function matrix of device, random character string alternatively code is generated in server end, retransmits option code and response request To client；Step 2, in client selection is gone out by option code decision-making, then draw corresponding in the Jacobian matrix element of selection Run time, then the run time of option code and corresponding element is sent to server；Step 3, in server end option code and run time are analyzed, calculate client running environment feature classification, judge It is not belonging in white list the client of running environment feature classification and to the client implementation access control policy.
- 5. a kind of anti-reptile method based on running environment feature recognition according to claim 4, it is characterised in that described Step 1, the cycle produce comprising selection Ziwen eigen selection subcharacter code.
- 6. a kind of anti-reptile method based on running environment feature recognition according to claim 5, it is characterised in that described Step 3, comprise the following steps：Step 3.1, inquiry server currently select subcharacter code, obtain it is currently used in Jacobian matrix and by analyze and based on Calculate the selection subvalue of initial setting up；Step 3.2, select current function matrix all option codes from client and corresponding in preset time section The run time of element, calculated by clustering algorithm, machine learning algorithm, algorithm for pattern recognition, deep neural network algorithm Client running environment feature classification；Step 3.3, judge to be not belonging to the client of running environment feature classification in white list and client mark is not conformed to Method and implementation access control policy.
- A kind of 7. anti-reptile based on running environment feature recognition according to any one claim in claim 4-6 Method, it is characterised in that described step 3, received by the memory module of server in preset time section from client The run time of option code and corresponding element.
- A kind of 8. anti-crawler system based on running environment feature recognition, it is characterised in that includingServer, including characteristics algorithm module, data interface module, memory module, data analysis module and access processing module, Characteristics algorithm module exports the strategy bag for updating and performing to server in itself；Client, the option code generated by characteristics algorithm module is received by data interface module；Described client is by option code implementation strategy bag；Described server, the characteristic by client executing strategy bag response is received by data interface module corresponding selection code According to data interface module also exports the characteristic of corresponding selection code to memory module, and data analysis module calculates storage mould The characteristic of corresponding selection code and by result of calculation feedback information to access processing module in block, access processing module is by feeding back Information is to the client executing predetermined policy.
- 9. a kind of anti-crawler system based on running environment feature recognition according to claim 8, it is characterised in that described Strategy bag, including with different run time program functions as the Jacobian matrix of element, for mapping character strings to function The selection of different elements is sub in matrix and includes the selection subcharacter code of selection Ziwen eigen.
- A kind of 10. server based on running environment feature recognition, with anti-reptile function, it is characterised in that includingCharacteristics algorithm module, cycle are produced with different run time program functions as the Jacobian matrix of element, for mapping Character string selection of different elements into Jacobian matrix is sub and includes the selection subcharacter code of selection Ziwen eigen；Data interface module, for client interaction data, export as caused by characteristics algorithm module option code to client And receive the run time from client corresponding selection code；Memory module, receive the number of the corresponding selection code run time received in data interface module scheduled time section According to；Data analysis module, calculate in memory module data in scheduled time section and the operation of client is judged by result of calculation Environmental characteristic classification；Access processing module, the client running environment feature classification calculated by data analysis module make a reservation for client executing Strategy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710203203.4A CN107426148B (en) | 2017-03-30 | 2017-03-30 | Crawler-resisting method and system based on running environment feature recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710203203.4A CN107426148B (en) | 2017-03-30 | 2017-03-30 | Crawler-resisting method and system based on running environment feature recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107426148A true CN107426148A (en) | 2017-12-01 |
| CN107426148B CN107426148B (en) | 2020-07-31 |
Family
ID=60423364
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710203203.4A Active CN107426148B (en) | 2017-03-30 | 2017-03-30 | Crawler-resisting method and system based on running environment feature recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107426148B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108133140A (en) * | 2017-12-08 | 2018-06-08 | 成都数聚城堡科技有限公司 | A kind of mode of the anti-reptile of dynamic |
| CN108521428A (en) * | 2018-04-20 | 2018-09-11 | 武汉极意网络科技有限公司 | A kind of realization method and system of the anti-reptile of public network based on jenkins |
| CN109815380A (en) * | 2018-12-20 | 2019-05-28 | 山东中创软件工程股份有限公司 | A kind of information crawler method, apparatus, equipment and computer readable storage medium |
| CN109818949A (en) * | 2019-01-17 | 2019-05-28 | 济南浪潮高新科技投资发展有限公司 | A kind of anti-crawler method neural network based |
| CN110096266A (en) * | 2019-05-13 | 2019-08-06 | 上海优扬新媒信息技术有限公司 | A kind of characteristic processing method and device |
| CN112312152A (en) * | 2020-10-27 | 2021-02-02 | 浙江集享电子商务有限公司 | Data processing system in network live broadcast |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104008339A (en) * | 2014-06-05 | 2014-08-27 | 东南大学 | Active technology based malicious code capture method |
| CN104539053A (en) * | 2014-12-31 | 2015-04-22 | 国家电网公司 | Power dispatching automation polling robot and method based on reptile technology |
| CN104618132A (en) * | 2014-12-16 | 2015-05-13 | 北京神州绿盟信息安全科技股份有限公司 | Generation method and generation device for application program recognition rule |
| CN105577701A (en) * | 2016-03-09 | 2016-05-11 | 携程计算机技术（上海）有限公司 | Web crawler recognition method and system |
| CN105743901A (en) * | 2016-03-07 | 2016-07-06 | 携程计算机技术（上海）有限公司 | Server, anti-crawler system and anti-crawler verification method |
| CN105812366A (en) * | 2016-03-14 | 2016-07-27 | 携程计算机技术（上海）有限公司 | Server, anti-crawler system and anti-crawler verification method |
| CN105871850A (en) * | 2016-04-05 | 2016-08-17 | 携程计算机技术（上海）有限公司 | Crawler detection method and crawler detection system |
| US9552489B1 (en) * | 2013-09-19 | 2017-01-24 | Imdb.Com, Inc. | Restricting network spidering |
| CN106534062A (en) * | 2016-09-23 | 2017-03-22 | 南京途牛科技有限公司 | Crawler prevention method |
-
2017
- 2017-03-30 CN CN201710203203.4A patent/CN107426148B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9552489B1 (en) * | 2013-09-19 | 2017-01-24 | Imdb.Com, Inc. | Restricting network spidering |
| CN104008339A (en) * | 2014-06-05 | 2014-08-27 | 东南大学 | Active technology based malicious code capture method |
| CN104618132A (en) * | 2014-12-16 | 2015-05-13 | 北京神州绿盟信息安全科技股份有限公司 | Generation method and generation device for application program recognition rule |
| CN104539053A (en) * | 2014-12-31 | 2015-04-22 | 国家电网公司 | Power dispatching automation polling robot and method based on reptile technology |
| CN105743901A (en) * | 2016-03-07 | 2016-07-06 | 携程计算机技术（上海）有限公司 | Server, anti-crawler system and anti-crawler verification method |
| CN105577701A (en) * | 2016-03-09 | 2016-05-11 | 携程计算机技术（上海）有限公司 | Web crawler recognition method and system |
| CN105812366A (en) * | 2016-03-14 | 2016-07-27 | 携程计算机技术（上海）有限公司 | Server, anti-crawler system and anti-crawler verification method |
| CN105871850A (en) * | 2016-04-05 | 2016-08-17 | 携程计算机技术（上海）有限公司 | Crawler detection method and crawler detection system |
| CN106534062A (en) * | 2016-09-23 | 2017-03-22 | 南京途牛科技有限公司 | Crawler prevention method |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108133140A (en) * | 2017-12-08 | 2018-06-08 | 成都数聚城堡科技有限公司 | A kind of mode of the anti-reptile of dynamic |
| CN108521428A (en) * | 2018-04-20 | 2018-09-11 | 武汉极意网络科技有限公司 | A kind of realization method and system of the anti-reptile of public network based on jenkins |
| CN108521428B (en) * | 2018-04-20 | 2020-09-01 | 武汉极意网络科技有限公司 | Realization method and system for preventing reptiles in public network based on jenkins |
| CN109815380A (en) * | 2018-12-20 | 2019-05-28 | 山东中创软件工程股份有限公司 | A kind of information crawler method, apparatus, equipment and computer readable storage medium |
| CN109818949A (en) * | 2019-01-17 | 2019-05-28 | 济南浪潮高新科技投资发展有限公司 | A kind of anti-crawler method neural network based |
| CN110096266A (en) * | 2019-05-13 | 2019-08-06 | 上海优扬新媒信息技术有限公司 | A kind of characteristic processing method and device |
| CN110096266B (en) * | 2019-05-13 | 2023-12-22 | 度小满科技(北京)有限公司 | Feature processing method and device |
| CN112312152A (en) * | 2020-10-27 | 2021-02-02 | 浙江集享电子商务有限公司 | Data processing system in network live broadcast |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107426148B (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107426148A (en) | A kind of anti-reptile method and system based on running environment feature recognition | |
| CN103297435B (en) | A kind of abnormal access behavioral value method and system based on WEB daily record | |
| CN105516113B (en) | System and method for automatic network fishing detected rule evolution | |
| CN111639363B (en) | Data analysis method based on block chain and edge computing server | |
| CN103685308B (en) | Detection method and system of phishing web pages, client and server | |
| CN106777024A (en) | Recognize the method and device of malicious user | |
| CN100456286C (en) | Universal file search system and method | |
| CN108156131A (en) | Webshell detection methods, electronic equipment and computer storage media | |
| CN110781308A (en) | Anti-fraud system for building knowledge graph based on big data | |
| CN107888606A (en) | A kind of domain name credit assessment and system | |
| CN104202291A (en) | Anti-phishing method based on multi-factor comprehensive assessment method | |
| Luo et al. | A CNN-based Approach to the Detection of SQL Injection Attacks | |
| CN108718341A (en) | Shared and search the method for data | |
| CN113761531A (en) | Malicious software detection system and method based on distributed API (application program interface) feature analysis | |
| CN109818949A (en) | A kind of anti-crawler method neural network based | |
| Xi et al. | A hybrid algorithm of traffic accident data mining on cause analysis | |
| CN110457576A (en) | Account-classification method, device, computer equipment and storage medium | |
| Sujatha | Improved user navigation pattern prediction technique from web log data | |
| Chen et al. | Network intrusion detection using class association rule mining based on genetic network programming | |
| CN110245986A (en) | A method of obtaining internet financial advertising body release | |
| CN107231364A (en) | A kind of website vulnerability detection method and device, computer installation and storage medium | |
| CN104657491B (en) | A kind of method that data are sent to client | |
| CN109948339A (en) | A kind of malicious script detection method based on machine learning | |
| CN111612531A (en) | Click fraud detection method and system | |
| CN112003884A (en) | Network asset acquisition and natural language retrieval method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |