CN106484720B - A kind of method and apparatus of the validity of detection popularization URL - Google Patents
A kind of method and apparatus of the validity of detection popularization URL Download PDFInfo
- Publication number
- CN106484720B CN106484720B CN201510541919.6A CN201510541919A CN106484720B CN 106484720 B CN106484720 B CN 106484720B CN 201510541919 A CN201510541919 A CN 201510541919A CN 106484720 B CN106484720 B CN 106484720B
- Authority
- CN
- China
- Prior art keywords
- url
- request
- head
- popularization
- counter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
- G06F16/9566—URL specific, e.g. using aliases, detecting broken or misspelled links
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses the method and apparatus that the validity of URL is promoted in a kind of detection, this method comprises: obtaining popularization URL to be detected；For each popularization URL to be detected, HEAD request first is initiated to popularization URL；If HEAD is requested successfully, it is determined that popularization URL is effective；If HEAD request failure, then GET request is initiated to popularization URL；If GET request success, it is determined that popularization URL is effective；If GET request fails, it is determined that popularization URL is invalid.Technical solution provided by the invention is by way of first sending HEAD request, the flow and bandwidth of request process are farthest saved as far as possible, further through the mode of a GET request additional after the HEAD of failure request, the erroneous judgement returned the result is requested to HEAD caused by eliminating in the case where promoting URL and HEAD may not be supported to request, further improves the efficiency, reliability and validity of detection.
Description
Technical field
The present invention relates to Internet technical fields, and in particular to a kind of method and apparatus of the validity of detection popularization URL.
Background technique
With the continuous development of Internet technology, Internet user is increasing, forms huge popularization audient, increasingly
More popularization sides with popularization demand wish to be promoted and publicized by internet, to improve popularization efficiency.Usual situation
Under, promotional content is distributed to viewer by way of URL link by popularization side, which characterizes in a specific popularization
Hold address on the internet, be referred to as " promoting URL ", when viewer, which clicks, promotes URL, corresponding targeted promotion content will
Display on a web browser, and is opened and is run according to the type of targeted promotion content, is realized and is pushed away to viewer
The scheme of wide content.
However, when the information for promoting the targeted promotion content for including in URL is abnormal, as information is expired or information deletion
When, will lead to viewer can not access popularization URL, i.e. popularization URL failure becomes the promotion link of failure.Greatly
Measure failure promotion link in vain occupy Internet resources while, be not only unable to satisfy the popularization demand of popularization side, also without
Method meets viewer to the browsing demand of promotional content.
Currently, thering is quite a few to promote the popularization URL of URL especially manual maintenance, only in the popularization URL of publication
The validity for promoting URL is detected by manually, the detection scheme efficiency is lower and testing result error is larger.
Summary of the invention
In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kind
The method and apparatus for stating the validity of detection popularization URL of problem a kind of.
According to one aspect of the present invention, a kind of method of the validity of detection popularization URL is provided, this method comprises:
Obtain popularization URL to be detected；
For each popularization URL to be detected, HEAD request first is initiated to popularization URL；
If HEAD is requested successfully, it is determined that popularization URL is effective；
If HEAD request failure, then GET request is initiated to popularization URL；If GET request success, it is determined that popularization URL
Effectively；If GET request fails, it is determined that popularization URL is invalid.
Optionally, this method further comprises:
HEAD is requested to fail, but GET request successfully promotes URL, counts the number of its HEAD request failure, if
The number of HEAD request failure is not up to preset value and is then still detected by the way of first sending HEAD request, when HEAD is requested
After the number of failure reaches preset value, detected by the way of directly transmitting GET request.
Optionally, this method further comprises:
It requests to fail for HEAD, but GET request successfully promotes URL and a counter is arranged；
The counter remembers a number when the HEAD about popularization URL requests failure；
When the counts of the counter are not up to preset value, if the HEAD of popularization URL is requested successfully, the meter
Number device resets.
Optionally, the counter is up counter, and the initial value of the counter is 0.
Optionally, the counter is subtraction count device, and the initial value of the counter is the preset value.
Optionally, this method further comprises:
It is detected by the way of directly transmitting GET request after requesting the number to fail to reach preset value HEAD
URL is promoted, after a preset time lapses, again using HEAD request is first initiated, if HEAD requests failure, then initiates GET request
Mode detected.
According to another aspect of the invention, a kind of device of the validity of detection popularization URL, the device packet are provided
It includes:
Acquiring unit is suitable for obtaining popularization URL to be detected；
Detection unit is suitable for for each popularization URL to be detected, first initiates HEAD request to popularization URL；
If HEAD is requested successfully, it is determined that popularization URL is effective；If HEAD request failure, then popularization URL is initiated
GET request；If GET request success, it is determined that popularization URL is effective；If GET request fails, it is determined that popularization URL is invalid.
Optionally, the detection unit is further adapted for that HEAD is requested to fail, but GET request is successfully promoted
URL counts the number of its HEAD request failure, still uses if the number of HEAD request failure is not up to preset value and first sends
The mode of HEAD request is detected, after the number of HEAD request failure reaches preset value, using directly transmitting GET request
Mode is detected.
Optionally, the detection unit is further adapted for HEAD request failure, but GET request is successfully promoted URL and set
Set a counter；The counter remembers a number when the HEAD about popularization URL requests failure；When the meter of the counter
When number is not up to preset value for several times, if the HEAD of popularization URL is requested successfully, the counter resets.
Optionally, the counter is up counter, and the initial value of the counter is 0.
Optionally, the counter is subtraction count device, and the initial value of the counter is the preset value.
Optionally, the detection unit is further adapted for using after requesting the number of failure to reach preset value HEAD
The popularization URL that the mode of GET request is detected is directly transmitted, after a preset time lapses, is asked again using first initiation HEAD
It asks, if HEAD request failure, then initiate the mode of GET request and detected.
It can be seen from the above, by promote URL push promotional content scene in, need to promote URL validity into
Row efficiently and accurately detects, to guarantee the validity of push；Technical solution provided by the invention is being not known by popularization to be detected
In the case that whether URL supports HEAD to request, for popularization URL to be detected, using HEAD request is first sent, if HEAD is requested
The validity of URL is promoted in the method detection for unsuccessfully retransmiting GET request, HEAD request and the request of any of GET request at
When function, determine that popularization URL is effective；When HEAD request and GET request fail, determine that popularization URL is invalid.The program is logical
After the mode for first sending HEAD request, the flow and bandwidth of request process are farthest saved as far as possible, further through in failure
HEAD request after an additional GET request mode, eliminating may not support HEAD to lead in the case where request promoting URL
What is caused requests the erroneous judgement returned the result to HEAD, further improves the efficiency, reliability and validity of detection.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows a kind of flow chart of the method for the validity of detection popularization URL according to an embodiment of the invention；
Fig. 2 shows a kind of data flow diagrams of the validity of detection popularization URL according to an embodiment of the invention；
Fig. 3 shows a kind of schematic diagram of the device of the validity of detection popularization URL according to an embodiment of the invention.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
In internet area, for popularization side using scene of the URL to viewer's push promotional content is promoted, to guarantee
The validity of push process needs periodic detection to promote the validity of URL, to avoid due to promoting URL failure and to popularization side
With the bring inconvenience of browsing side and loss.Based on this, the method for the validity of URL is promoted the present invention provides a kind of detection and is
System, will hereafter be described in detail it by specific embodiment.
Fig. 1 shows a kind of flow chart of the method for the validity of detection popularization URL according to an embodiment of the invention.
As shown in Figure 1, this method comprises:
Step S110 obtains popularization URL to be detected.
Step S120 first initiates HEAD request to popularization URL for each popularization URL to be detected.
Step S130, if HEAD is requested successfully, it is determined that popularization URL is effective.
Step S140, if HEAD request failure, then GET request is initiated to popularization URL.
Step S150, if GET request success, it is determined that popularization URL is effective.
Step S160, if GET request fails, it is determined that popularization URL is invalid.
In the above process, the HEAD request and GET request are the side interacted defined in http protocol with server
Method.Wherein, HEAD request is the request to resource header message, and the data volume of return is smaller, and GET request is to resource entirety
The data volume of the request of information, return is larger, and in contrast, in the case where not needing to obtain resource Global Information, HEAD is asked
It asks to be more advantageous to and saves flow and bandwidth；But since GET request is the basic request method in http protocol, then for
For the popularization URL of detection, default supports GET request, but not necessarily supports HEAD request.
Therefore, method shown in FIG. 1 be not known by popularization URL to be detected whether support HEAD request in the case where, for
Popularization URL to be detected, using HEAD request is first sent, if the method detection that HEAD request unsuccessfully retransmits GET request is promoted
The validity of URL determines that popularization URL is effective when any of HEAD request and GET request are requested successfully；It is asked in HEAD
When GET request of summing fails, determine that popularization URL is invalid.The program is by way of first sending HEAD request, as far as possible most
Save to big degree the flow and bandwidth of request process；Further through the side of a GET request additional after the HEAD of failure request
Formula requests the erroneous judgement returned the result to HEAD caused by eliminating in the case where promoting URL and HEAD may not be supported to request, into
One step improves the efficiency, reliability and validity of detection.
Under normal conditions, in method shown in Fig. 1, after being detected to popularization URL, if testing result is that HEAD is asked
Ask failure and GET request success, then can learn: popularization URL is effective, and popularization URL do not support HEAD request or
The HEAD request of popularization URL is shielded, i.e. the HEAD request of popularization URL is meaningless.For having determined that HEAD request is not intended to
The popularization URL of justice sends HEAD request to it without repeating when carrying out repeating to detect to its validity.
For this purpose, in one embodiment of the invention, method shown in FIG. 1 further comprises: HEAD is requested to fail,
But GET request successfully promotes URL, counts the number of its HEAD request failure, if the number of HEAD request failure is not up to
Preset value is then still detected by the way of first sending HEAD request, after the number of HEAD request failure reaches preset value,
It is detected by the way of directly transmitting GET request.Wherein, the purpose that preset value is arranged is to eliminate to be caused by uncertain factor
Single error caused by influence.
Specifically, the HEAD request frequency of failure can be counted in the following way: requesting to fail for HEAD, but GET request
It successfully promotes URL and one counter is set；Counter note is primary when the HEAD about popularization URL requests failure
Number；When the counts of the counter are not up to preset value, if the HEAD of popularization URL is requested successfully, the counter
It resets.Wherein, the counter is up counter, and the initial value of the counter is 0；The counter may be subtraction meter
Number device, the initial value of the counter are the preset value.
For example, the initial value of setting subtraction count device is 128, in the validity of repeated detection one specified popularization URL
During, whenever obtain HEAD request failure as a result, just subtraction count device is enabled to subtract 1；It is reached in the value of subtraction count device
It is successful as a result, subtraction count device reset is just enabled to return to 128 whenever obtaining a HEAD request to before 0；Under this logic,
Until the value of subtraction count device is 0, determine that the corresponding URL that promotes does not support HEAD to request, from having for detection popularization URL next time
Effect property starts, and is detected by the way of directly transmitting GET request.
Further, in order to avoid being likely to occur the erroneous judgement to promoting URL HEAD not being supported to request in the above process,
On the basis of above-mentioned, method shown in FIG. 1 further include: use and directly transmit after requesting the number to fail to reach preset value HEAD
The popularization URL that the mode of GET request is detected, after a preset time lapses, again using HEAD request is first initiated, if HEAD
Request failure, then initiate the mode of GET request and detected.
Fig. 2 shows a kind of data flow diagrams of the validity of detection popularization URL according to an embodiment of the invention.
As shown in Fig. 2, one or more task agent devices obtain popularization URL to be detected, for each popularization URL of acquisition, according to pre-
If strategy selects a detection service device from detection service device cluster, popularization URL is sent to the detection service of the selection
Device；The popularization URL received is distributed on multiple crawler schedulers by each detection service device；The popularization that each crawler scheduler will receive
URL is distributed to corresponding multiple crawlers, by crawlers to the popularization URL received using first transmission HEAD request, if
The mode that HEAD request unsuccessfully retransmits GET request is detected, the testing result of the validity for the URL that puts it over；In addition, each
The testing result of the validity of the popularization URL of acquisition is returned to corresponding crawler scheduler by crawlers, then is dispatched by crawler
Device returns to corresponding detection service device.
It should be noted that may include one or more task agent devices in Fig. 2, the status of multiple tasks proxy server is
Of equal value, and the execution logic of each task agent device therein and a task agent device is consistent, therefore, in Fig. 2 only
One task agent device is shown to be illustrated as representative；Each detection service device and crawler scheduler in detection service device cluster
Interactive relation correspond to identical, each crawler scheduler also corresponds to identical with the interactive relation of crawlers；Therefore, only show in Fig. 2
The interactive relation of first detection service device and multiple crawler schedulers is illustrated and first crawler tune as representing out
Degree device and the interactive relation of multiple crawlers are illustrated as representative.
It in the present embodiment, include multiple detection service device groups in detection service device cluster shown in Fig. 2, each detection clothes
It include multiple detection service devices in device group of being engaged in；Then task agent device for acquisition each popularization URL, according to popularization URL's
The cryptographic Hash of domain name selects a detection service device group from detection service device cluster, then from the detection service device group of selection
Select a detection service device.Setting in this way, a large amount of popularization URL to be detected are dispersed in multiple detection service device groups
Multiple detection service devices on handled, significantly reduce the detection pressure of each detection service device, improve detection efficiency, reduce
Detect rate of breakdown.
Specifically, it includes: to take to the detection that a detection service device is selected in above-mentioned detection service device group from selection
Each detection service device in device group of being engaged in sends Ping request, obtains the current response time of each detection service device, selects current ring
Shortest detection service device between seasonable, i.e., a detection service for selecting current response most fast from the detection service device of selection
Device, for receiving current popularization URL to be detected.In some cases, each detection clothes above-mentioned into the detection service device group
Device of being engaged in sends Ping request, during the current response time for obtaining each detection service device, often due to network jitter and
Cause the current response time of each detection service device unstable, for this purpose, after the current response time for obtaining each detection service device,
By shortest detection service device of current response time with described in preceding one-time detection promote URL when select detection service device it is current
Response time is compared, if the two gap is more than preset range, selects shortest detection service device of current response time；
If the two gap is less than preset range, the detection service device selected when promoting URL described in preceding one-time detection is continued to use.
For example, each detection service device of the task agent device into a detection service device group sends Ping request, obtain each
The response time of detection service device, in a upper period, response time shortest detection service device is A detection service device, and
In current slot, response time shortest detection service device is B server, to influence caused by removal network jitter, is preset
T0, by the current response time T of A detection service deviceAWith the current response time T of B detection service deviceBIt is compared, if TA-TB
≥T0, then show that the performance of the B detection service device in current slot is better than A detection service device really, therefore select B detection clothes
Business device；If TA-TB<T0, then it is possible to show that the response speed of the B detection service device in current slot is faster than A detection service device
It is that not can determine that the performance of B detection service device is better than A detection service device really, therefore still continue to use A caused by network jitter
Detection service device.
In the present embodiment, the testing result of the popularization URL validity obtained in each crawlers will finally return to phase
After the detection service device answered, further, all detections clothes in the detection service device group where corresponding detection service device
Being engaged in, device is synchronous to save the testing result for promoting URL validity returned.Setting in this way, so that the same detection service device
All detection service devices in group can share the obtained detection knot for promoting URL validity of any one detection service device
Fruit, the state of synchronized update in holding group, when calling any one detection service device in the same detection service device group
Know history testing result, repeats detection without carrying out to identical popularization URL.
In the present embodiment, crawler scheduler can be the module in detection service device, be also possible on hardware independently of inspection
Survey the machine of server, each detection service when crawler scheduler is the machine on hardware independently of detection service device, in Fig. 2
The popularization URL received is distributed on multiple crawler schedulers by device include: each detection service device by each popularization URL received into
Row dns resolution obtains the corresponding IP address of popularization URL, popularization URL is distributed to and belongs to areal with its IP address
On crawler scheduler.Wherein, each detection service device can monitor the operation conditions of multiple crawler schedulers, when a crawler tune
Degree device is when breaking down, detection service device can shift the popularization URL to be detected on the crawler scheduler distribute near its
His crawler scheduler.
In the present embodiment, each crawlers have status of equal value, each principle of the crawler scheduler based on load balancing
The popularization URL received is distributed to corresponding multiple crawlers, i.e., the popularization URL received is equally distributed to multiple crawlers
Program.
Fig. 3 shows a kind of schematic diagram of the device of the validity of detection popularization URL according to an embodiment of the invention,
As shown in figure 3, the device 300 that the validity of URL is promoted in the detection includes:
Acquiring unit 310 is suitable for obtaining popularization URL to be detected.
Detection unit 320 is suitable for for each popularization URL to be detected, first initiates HEAD request to popularization URL；If
HEAD is requested successfully, it is determined that popularization URL is effective；If HEAD request failure, then GET request is initiated to popularization URL；If
GET request success, it is determined that popularization URL is effective；If GET request fails, it is determined that popularization URL is invalid.
In above-mentioned apparatus description, the HEAD request and GET request are to interact defined in http protocol with server
Method.Wherein, HEAD request is the request to resource header message, and the data volume of return is smaller, and GET request is to resource
The data volume of the request of Global Information, return is larger, in contrast, in the case where not needing to obtain resource Global Information,
HEAD request, which is more advantageous to, saves flow and bandwidth；But since GET request is the basic request method in http protocol, then
For popularization URL to be detected, default supports GET request, but not necessarily supports HEAD request.
Therefore, device shown in Fig. 3 be not known by popularization URL to be detected whether support HEAD request in the case where, for
Popularization URL to be detected, using HEAD request is first sent, if the method detection that HEAD request unsuccessfully retransmits GET request is promoted
The validity of URL determines that popularization URL is effective when any of HEAD request and GET request are requested successfully；It is asked in HEAD
When GET request of summing fails, determine that popularization URL is invalid.The program is by way of first sending HEAD request, as far as possible most
Save to big degree the flow and bandwidth of request process；Further through the side of a GET request additional after the HEAD of failure request
Formula requests the erroneous judgement returned the result to HEAD caused by eliminating in the case where promoting URL and HEAD may not be supported to request, into
One step improves the efficiency, reliability and validity of detection.
In one embodiment of the invention, the detection unit 320 of Fig. 3 shown device, is further adapted for asking HEAD
Failure is asked, but GET request successfully promotes URL, count the number of its HEAD request failure, if the number of HEAD request failure
Not up to preset value is then still detected by the way of first sending HEAD request, when the number of HEAD request failure reaches default
After value, detected by the way of directly transmitting GET request.
Specifically, the detection unit 320 of Fig. 3 shown device, be further adapted for HEAD request failure, but GET request at
A counter is arranged in the popularization URL of function；The counter remembers a number when the HEAD about popularization URL requests failure；
When the counts of the counter are not up to preset value, if the HEAD of popularization URL is requested successfully, the counter is multiple
Position.Wherein, the counter is up counter, and the initial value of the counter is 0；The counter is subtraction count device, should
The initial value of counter is the preset value.
Further, in order to avoid being likely to occur the erroneous judgement to promoting URL HEAD not being supported to request in the above process,
On the basis of above-mentioned, detection unit 320 is further adapted for after requesting the number of failure to reach preset value HEAD using directly hair
The popularization URL for sending the mode of GET request to be detected, after a preset time lapses, again using first initiation HEAD request, if
HEAD request failure, then initiate the mode of GET request and detected.
It should be noted that each embodiment and each embodiment shown in figure 1 above-Fig. 2 of Fig. 3 shown device are corresponding identical,
It has been described in detail above, details are not described herein.
In conclusion needing to carry out the validity for promoting URL in the scene for pushing promotional content by promoting URL
It efficiently and accurately detects, to guarantee the validity of push；Technical solution provided by the invention is being not known by popularization URL to be detected
In the case where whether supporting HEAD to request, for popularization URL to be detected, using HEAD request is first sent, if HEAD request is lost
The validity for retransmiting the method detection popularization URL of GET request is lost, is requested successfully in any of HEAD request and GET request
When, determine that popularization URL is effective；When HEAD request and GET request fail, determine that popularization URL is invalid.The program passes through
The mode for first sending HEAD request, farthest saves the flow and bandwidth of request process as far as possible；Further through in failure
The mode of an additional GET request after HEAD request, eliminating causes in the case where promoting URL and may not support HEAD request
The erroneous judgement that returns the result is requested to HEAD, further improve detection and promote the efficiency of validity of URL, reliability and effectively
Property, it ensure that the validity of the push process of promotional content, can more ideally meet the popularization demand in current internet field.
It should be understood that
Algorithm and display be not inherently related to any certain computer, virtual bench or other equipment provided herein.
Various fexible units can also be used together with teachings based herein.As described above, it constructs required by this kind of device
Structure be obvious.In addition, the present invention is also not directed to any particular programming language.It should be understood that can use various
Programming language realizes summary of the invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention
Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this specification.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of the various inventive aspects,
Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the disclosed method should not be interpreted as reflecting the following intention: i.e. required to protect
Shield the present invention claims features more more than feature expressly recited in each claim.More precisely, as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim itself
All as a separate embodiment of the present invention.
Those skilled in the art will understand that can be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more devices different from this embodiment.It can be the module or list in embodiment
Member or component are combined into a module or unit or component, and furthermore they can be divided into multiple submodule or subelement or
Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it can use any
Combination is to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed
All process or units of what method or apparatus are combined.Unless expressly stated otherwise, this specification is (including adjoint power
Benefit require, abstract and attached drawing) disclosed in each feature can carry out generation with an alternative feature that provides the same, equivalent, or similar purpose
It replaces.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed
Meaning one of can in any combination mode come using.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors
Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice
Microprocessor or digital signal processor (DSP) realize that the validity of URL is promoted in detection according to an embodiment of the present invention
The some or all functions of some or all components in device.The present invention is also implemented as being retouched here for executing
The some or all device or device programs (for example, computer program and computer program product) for the method stated.
It is such to realize that program of the invention can store on a computer-readable medium, or can have one or more signal
Form.Such signal can be downloaded from an internet website to obtain, be perhaps provided on the carrier signal or with it is any its
He provides form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability
Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real
It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch
To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame
Claim.
The invention discloses a kind of methods that the validity of URL is promoted in A1, detection, wherein this method comprises:
Obtain popularization URL to be detected；
For each popularization URL to be detected, HEAD request first is initiated to popularization URL；
If HEAD is requested successfully, it is determined that popularization URL is effective；
If HEAD request failure, then GET request is initiated to popularization URL；If GET request success, it is determined that popularization URL
Effectively；If GET request fails, it is determined that popularization URL is invalid.
A2, method as described in a1, wherein this method further comprises:
HEAD is requested to fail, but GET request successfully promotes URL, counts the number of its HEAD request failure, if
The number of HEAD request failure is not up to preset value and is then still detected by the way of first sending HEAD request, when HEAD is requested
After the number of failure reaches preset value, detected by the way of directly transmitting GET request.
A3, as described in A2 method, wherein this method further comprises:
It requests to fail for HEAD, but GET request successfully promotes URL and a counter is arranged；
The counter remembers a number when the HEAD about popularization URL requests failure；
When the counts of the counter are not up to preset value, if the HEAD of popularization URL is requested successfully, the meter
Number device resets.
A4, the method as described in A3, wherein
The counter is up counter, and the initial value of the counter is 0.
A5, the method as described in A3, wherein
The counter is subtraction count device, and the initial value of the counter is the preset value.
A6, as described in A2 method, wherein this method further comprises:
It is detected by the way of directly transmitting GET request after requesting the number to fail to reach preset value HEAD
URL is promoted, after a preset time lapses, again using HEAD request is first initiated, if HEAD requests failure, then initiates GET request
Mode detected.
The invention also discloses the devices that the validity of URL is promoted in B7, a kind of detection, wherein the device includes:
Acquiring unit is suitable for obtaining popularization URL to be detected；
Detection unit is suitable for for each popularization URL to be detected, first initiates HEAD request to popularization URL；
If HEAD is requested successfully, it is determined that popularization URL is effective；If HEAD request failure, then popularization URL is initiated
GET request；If GET request success, it is determined that popularization URL is effective；If GET request fails, it is determined that popularization URL is invalid.
B8, device as described in b7, wherein
The detection unit is further adapted for that HEAD is requested to fail, but GET request successfully promotes URL, counts it
The number of HEAD request failure still uses if the number of HEAD request failure is not up to preset value and first sends HEAD request
Mode is detected, and after the number of HEAD request failure reaches preset value, is examined by the way of directly transmitting GET request
It surveys.
B9, the device as described in B8, wherein
The detection unit is further adapted for HEAD request failure, but GET request successfully promotes URL and is arranged one
Counter；The counter remembers a number when the HEAD about popularization URL requests failure；When the counts of the counter
Not up to preset value when, if the HEAD of popularization URL is requested successfully, the counter resets.
B10, the device as described in B9, wherein
The counter is up counter, and the initial value of the counter is 0.
B11, the device as described in B9, wherein
The counter is subtraction count device, and the initial value of the counter is the preset value.
B12, the device as described in B8, wherein
The detection unit is further adapted for requesting HEAD the number of failure to reach after preset value using directly transmitting
The popularization URL that the mode of GET request is detected, after a preset time lapses, again using HEAD request is first initiated, if HEAD
Request failure, then initiate the mode of GET request and detected.
Claims (10)
1. a kind of method that the validity of URL is promoted in detection, wherein this method comprises:
Obtain popularization URL to be detected；
For each popularization URL to be detected, HEAD request first is initiated to popularization URL；
If HEAD is requested successfully, it is determined that popularization URL is effective；
If HEAD request failure, then GET request is initiated to popularization URL；If GET request success, it is determined that popularization URL has
Effect；If GET request fails, it is determined that popularization URL is invalid；
This method further comprises:
HEAD is requested to fail, but GET request successfully promotes URL, the number of its HEAD request failure is counted, if HEAD
The number of request failure is not up to preset value and is then still detected by the way of first sending HEAD request, when HEAD requests to fail
Number reach preset value after, detected by the way of directly transmitting GET request.
2. the method for claim 1, wherein this method further comprises:
It requests to fail for HEAD, but GET request successfully promotes URL and a counter is arranged；
The counter remembers a number when the HEAD about popularization URL requests failure；
When the counts of the counter are not up to preset value, if the HEAD of popularization URL is requested successfully, the counter
It resets.
3. method according to claim 2, wherein
The counter is up counter, and the initial value of the counter is 0.
4. method according to claim 2, wherein
The counter is subtraction count device, and the initial value of the counter is the preset value.
5. the method for claim 1, wherein this method further comprises:
The popularization detected by the way of directly transmitting GET request after preset value is reached for the number of HEAD request failure
URL again using HEAD request is first initiated, if HEAD request fails, then initiates the side of GET request after a preset time lapses
Formula is detected.
6. the device that the validity of URL is promoted in a kind of detection, wherein the device includes:
Acquiring unit is suitable for obtaining popularization URL to be detected；
Detection unit is suitable for for each popularization URL to be detected, first initiates HEAD request to popularization URL；
If HEAD is requested successfully, it is determined that popularization URL is effective；If HEAD request failure, then GET is initiated to popularization URL and is asked
It asks；If GET request success, it is determined that popularization URL is effective；If GET request fails, it is determined that popularization URL is invalid；
The detection unit is further adapted for that HEAD is requested to fail, but GET request successfully promotes URL, counts its HEAD
The number for requesting failure, if the number of HEAD request failure is not up to preset value still by the way of first sending HEAD request
It is detected, after the number of HEAD request failure reaches preset value, is detected by the way of directly transmitting GET request.
7. device as claimed in claim 6, wherein
The detection unit is further adapted for HEAD request failure, but GET request successfully promotes URL and a counting is arranged
Device；The counter remembers a number when the HEAD about popularization URL requests failure；When the counts of the counter do not reach
When to preset value, if the HEAD of popularization URL is requested successfully, the counter resets.
8. device as claimed in claim 7, wherein
The counter is up counter, and the initial value of the counter is 0.
9. device as claimed in claim 7, wherein
The counter is subtraction count device, and the initial value of the counter is the preset value.
10. device as claimed in claim 6, wherein
The detection unit is further adapted for requesting HEAD the number of failure to reach after preset value using directly transmitting GET
The popularization URL that the mode of request is detected, after a preset time lapses, again using HEAD request is first initiated, if HEAD is asked
Failure is asked, then initiates the mode of GET request and is detected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510541919.6A CN106484720B (en) | 2015-08-28 | 2015-08-28 | A kind of method and apparatus of the validity of detection popularization URL |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510541919.6A CN106484720B (en) | 2015-08-28 | 2015-08-28 | A kind of method and apparatus of the validity of detection popularization URL |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106484720A CN106484720A (en) | 2017-03-08 |
| CN106484720B true CN106484720B (en) | 2019-07-23 |
Family
ID=58235069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510541919.6A Active CN106484720B (en) | 2015-08-28 | 2015-08-28 | A kind of method and apparatus of the validity of detection popularization URL |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106484720B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107368399A (en) * | 2017-06-28 | 2017-11-21 | 武汉斗鱼网络科技有限公司 | Webpage monitoring method and system on a kind of line |
| CN108011934B (en) * | 2017-11-24 | 2021-10-01 | 聚好看科技股份有限公司 | Method and device for processing resource data |
| CN109033399B (en) * | 2018-08-02 | 2021-06-18 | 挖财网络技术有限公司 | Method for detecting validity of link |
| CN109816453B (en) * | 2019-01-31 | 2021-04-16 | 珠海天燕科技有限公司 | Detection method and device for promotion resource link |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101350822A (en) * | 2008-09-08 | 2009-01-21 | 南开大学 | Method for discovering and tracing Internet malevolence code |
| CN103514206A (en) * | 2012-06-26 | 2014-01-15 | 百度在线网络技术（北京）有限公司 | Method and device for determining popularization configuration information corresponding to objective popularization object |
| CN104679798A (en) * | 2013-12-03 | 2015-06-03 | 腾讯科技（深圳）有限公司 | Webpage detection method and device |
| CN104769587A (en) * | 2012-10-30 | 2015-07-08 | 高通股份有限公司 | A preemptive framework for accessing short urls |
-
2015
- 2015-08-28 CN CN201510541919.6A patent/CN106484720B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101350822A (en) * | 2008-09-08 | 2009-01-21 | 南开大学 | Method for discovering and tracing Internet malevolence code |
| CN103514206A (en) * | 2012-06-26 | 2014-01-15 | 百度在线网络技术（北京）有限公司 | Method and device for determining popularization configuration information corresponding to objective popularization object |
| CN104769587A (en) * | 2012-10-30 | 2015-07-08 | 高通股份有限公司 | A preemptive framework for accessing short urls |
| CN104679798A (en) * | 2013-12-03 | 2015-06-03 | 腾讯科技（深圳）有限公司 | Webpage detection method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106484720A (en) | 2017-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106484720B (en) | A kind of method and apparatus of the validity of detection popularization URL | |
| EP3590248B1 (en) | Data loss prevention using category-directed parsers | |
| US10171299B2 (en) | Method and apparatus for configuring proxy server | |
| CN108681777B (en) | Method and device for running machine learning program based on distributed system | |
| WO2016181383A2 (en) | System and method for streaming content from multiple servers | |
| US10795744B2 (en) | Identifying failed customer experience in distributed computer systems | |
| CN103605689B (en) | It is a kind of to obtain the method and device for accessing the residence time | |
| CN105208108B (en) | File upload/method for down loading and system, server, client under Web environment | |
| CN103942225A (en) | Method and system for invoking resources of Hybrid App client and client | |
| US8346784B1 (en) | Java script reductor | |
| US20170126789A1 (en) | Automatic Software Controller Configuration based on Application and Network Data | |
| US20170168881A1 (en) | Process chain discovery across communication channels | |
| CN103825783A (en) | Test method and device | |
| CN110266737A (en) | A kind of leak detection method, device, equipment and medium that cross-domain resource is shared | |
| CN104967644A (en) | Message push method, apparatus and system | |
| CN110113440A (en) | A kind of request responding method, device and terminal | |
| CN106302712B (en) | Handle the method and device of web data | |
| CN110913270A (en) | Live broadcasting method and device | |
| CN109495319B (en) | Method, device and equipment for determining fault information of CDN node | |
| CN103051722B (en) | A kind ofly determine the method whether page is held as a hostage and relevant device | |
| US9729404B2 (en) | Quality of service monitoring device and method of monitoring quality of service | |
| CN108337227B (en) | Method and middleware based on OpenID account login application program | |
| CN109327530B (en) | Information processing method, device, electronic equipment and storage medium | |
| US20150082402A1 (en) | System and method for automated authentication | |
| CN110113412A (en) | A kind of method of data synchronization and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220706 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |