CN109495568A

CN109495568A - A kind of data capture method, data acquisition facility, server and storage medium

Info

Publication number: CN109495568A
Application number: CN201811377278.5A
Authority: CN
Inventors: 王志国
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd; Beijing Kingsoft Cloud Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd; Beijing Kingsoft Cloud Technology Co Ltd
Priority date: 2018-11-19
Filing date: 2018-11-19
Publication date: 2019-03-19
Anticipated expiration: 2038-11-19
Also published as: CN109495568B

Abstract

The embodiment of the invention provides a kind of data capture method, data acquisition facility, server and storage mediums, and wherein method comprises determining that the first data transmission index of each upper layer node of current detection cycle；For each upper layer node in each upper layer node, pretreatment and disposal of stability are carried out to the first data transmission index of upper layer node, obtain the second data transmission objective of upper layer node；The size of the second data transmission objective based on each upper layer node, is ranked up each upper layer node；Into each upper layer node after sequence, the upper layer node of position sequence first sends data acquisition request, so that the upper layer node of position sequence first returns to request data corresponding with data acquisition request.The embodiment of the present invention can be improved the Ranking Stability of each upper layer node.

Description

A kind of data capture method, data acquisition facility, server and storage medium

Technical field

The present invention relates to technical field of data transmission, more particularly to a kind of data capture method, data acquisition facility, clothes Business device and storage medium.

Background technique

The fast development of Internet technology is that user brings great convenience, but simultaneously user to the service quality of internet and Transmission speed requirement is also higher and higher, although this results in network bandwidth constantly increasing, as number of users is continuously increased, It is influenced by factors such as the load of Web server and transmission ranges, data transmission bauds is still partially slow.And CDN (Content Delivery Network, content distributing network) technology appearance, network data transmission quality can be optimized, to improve number According to transmission speed.

As shown in Figure 1, being usually deployed multiple fringe nodes and multiple upper layer nodes in CDN, wherein fringe node can To refer to the cache server being located near user, it is usually deployed in the network edge of entire CDN, for being obtained from upper layer node Data simultaneously cache；Upper layer node can refer to source server, the data center for being deployed in entire CDN is usually layered, for being responsible for The data that caching Hui Yuan is pulled.

When the data that the fringe node of CDN does not cache, just need to obtain data, data acquisition mistake from upper layer node Journey are as follows: fringe node can be detected periodically and the data transmission period of more each upper layer node is as data transmission objective, and Height based on data transmission objective is ranked up each upper layer node, and then obtains number from the upper layer node of position sequence first According to.

Fringe node in the related technology, the original data transmissions index due to being based on directly on each upper layer node are arranged Sequence, once this will lead to detecting periodically to the original data transmissions index of each upper layer node change, then each upper layer The position sequence of node will also fluctuate therewith, and it is all different to may cause the upper layer node that the fringe node selects every time.However in fact, It is not the required data that the fringe node is stored in each upper layer node, once the upper layer node of fringe node selection In it is not stored needed for data, then upper layer node needs data needed for further requesting again to a node layer thereon, then another layer one Layer is back to the fringe node, causes data acquisition process to increase, data obtaining time is caused to extend.That is, due to related skill The Ranking Stability of each upper layer node is poor in art, so as to cause fringe node from upper layer node obtain data when speed vulnerable to To influence.

Summary of the invention

The embodiment of the present invention is designed to provide a kind of data capture method, data acquisition facility, server and storage Medium, to improve the Ranking Stability of each upper layer node.Specific technical solution is as follows:

In a first aspect, the embodiment of the invention provides a kind of data capture methods, comprising:

Determine the first data transmission index of each upper layer node of current detection cycle；

For each upper layer node in each upper layer node, to the first data transmission index of the upper layer node Pretreatment and disposal of stability are carried out, the second data transmission objective of the upper layer node is obtained；

The size of the second data transmission objective based on each upper layer node, to each upper layer node into Row sequence；

Into each upper layer node after sequence, the upper layer node of position sequence first sends data acquisition request, so that position The upper layer node of sequence first returns to request data corresponding with the data acquisition request.

Optionally, the first data transmission index to the upper layer node carries out at pretreatment and stability Reason, the second data transmission objective for obtaining the upper layer node include:

The first data transmission index of the upper layer node is normalized, the upper layer node is obtained and exists The third data transmission objective of current detection cycle；

Accumulation process is carried out to all third data transmission objectives of the upper layer node in predetermined time period, Obtain the 4th data transmission objective of the upper layer node；Wherein, the predetermined time period includes current detection cycle and more A history detection cycle；

Disposal of stability is carried out to the 4th data transmission objective of the upper layer node, obtains the upper layer node Second data transmission objective.

Optionally, the first data transmission index to the upper layer node is normalized, and obtains institute Upper layer node is stated the third data transmission objective of current detection cycle the step of, comprising:

Based on the first default expression formula, the first data transmission index of upper layer node is normalized；

Wherein, the described first default expression formula are as follows:

In formula,Indicate the third data transmission objective of i-th of upper layer node in each upper layer node；t_iIndicate institute State the first data transmission index of i-th of upper layer node；t_minIndicate that the first data transmission index in all upper layer nodes is minimum First data transmission index；I is the integer greater than 0.

Optionally, all third data transmission objectives to the upper layer node in predetermined time period into Row accumulation process, the step of obtaining the 4th data transmission objective of the upper layer node, comprising:

By the second default expression formula, all third data transmission of the upper layer node in predetermined time period are referred to Mark carries out accumulation process；

Wherein, the described second default expression formula are as follows:

In formula,Indicate the 4th data transmission objective of i-th of upper layer node in each upper layer node；Period table Show predetermined time period；J indicates the detection cycle number that the predetermined time period includes；α indicates the first parameter preset；Indicate the third data transmission objective of i-th of upper layer node；I is the integer greater than 0.

Optionally, the 4th data transmission objective to the upper layer node carries out disposal of stability, obtains institute The step of stating the second data transmission objective of upper layer node, comprising:

Expression formula is preset by third, disposal of stability is carried out to the 4th data transmission objective of upper layer node；

Wherein, the third presets expression formula are as follows:

In formula,Indicate the second data transmission objective of i-th of upper layer node in each upper layer node；X indicates pre- If the truth of a matter；Indicate the 4th data transmission objective of i-th of upper layer node；I is the integer greater than 0.

Optionally, disposal of stability is carried out in the 4th data transmission objective to the upper layer node, obtained After second data transmission objective of the upper layer node, comprising:

Judge the row of the upper layer node each upper layer node in a preceding detection cycle for the current detection cycle In sequence, if be located at first sequence；

In the case where the upper layer node is not located at first sequence, by the upper layer node in current detection cycle Obtained in the second data transmission objective be weighted processing, obtain weighting treated the second data transmission objective；

In the case where the upper layer node is located at first sequence, retain the upper layer node in current detection cycle Obtained in the second data transmission objective.

Optionally, the size of the second data transmission objective based on each upper layer node, to each institute State the step of upper layer node is ranked up, comprising:

Determine the size relation of the specified data transmission objective in each upper layer node；Finger in the upper layer node Fixed number is according to transmission objective are as follows: the second data transmission objective of the weighted processing or second data retained transmission refer to Mark；

According to identified size relation, each upper layer node is ranked up.

Optionally, it is described by the upper layer node the second data transmission objective obtained in current detection cycle into Row weighting processing, the step of obtaining weighting treated the second data transmission objective, comprising:

By the 4th default expression formula, to each upper layer node in a preceding detection cycle for the current detection cycle In sequence, the second data transmission objective for being not located at the upper layer node of first sequence is weighted processing；

The 4th default expression formula are as follows:

In formula, m indicates weighting coefficient；Count_iIndicate that i-th of upper layer node is when described default in each upper layer node Between be located at the number of first sequence in length；Count_*It indicates to be located in a preceding detection cycle for the current detection cycle The upper layer node of first sequence is in the total degree that the predetermined time period internal sort is first sequence；β indicates the second default ginseng Number；γ indicates third parameter preset；I is the integer greater than 0.

Second aspect, the embodiment of the invention provides a kind of data acquisition facilities, comprising:

Determining module, the first data transmission index of each upper layer node for determining current detection cycle；

Processing module, for for each upper layer node in each upper layer node, to described the of the upper layer node One data transmission objective carries out pretreatment and disposal of stability, obtains the second data transmission objective of the upper layer node；

Sorting module, for the size of the second data transmission objective based on each upper layer node, to each The upper layer node is ranked up；

Sending module, the upper layer node for the position sequence first into each upper layer node after sequence send data and obtain Request is taken, so that the upper layer node of position sequence first returns to request data corresponding with the data acquisition request.

Optionally, the processing module, comprising:

Place is normalized for the first data transmission index to the upper layer node in normalized submodule Reason, obtains the upper layer node in the third data transmission objective of current detection cycle；

Accumulation process submodule, for being passed to all third data of the upper layer node in predetermined time period Defeated index carries out accumulation process, obtains the 4th data transmission objective of the upper layer node；Wherein, the predetermined time period packet Include current detection cycle and multiple history detection cycles；

Disposal of stability submodule carries out at stability for the 4th data transmission objective to the upper layer node Reason, obtains the second data transmission objective of the upper layer node.

Optionally, the normalized submodule, is used for:

Wherein, the described first default expression formula are as follows:

Optionally, the accumulation process submodule, is used for:

Wherein, the described second default expression formula are as follows:

Optionally, the disposal of stability submodule, is specifically used for:

The third presets expression formula are as follows:

Optionally, described device further include:

Judgment module, for judging that the upper layer node is each in a preceding detection cycle for the current detection cycle In the sequence of upper layer node, if be located at first sequence；

Processing module is weighted, in the case where the upper layer node is not located at first sequence, by the upper layer Node the second data transmission objective obtained in current detection cycle is weighted processing, obtains weighting treated the Two data transmission objectives；

Reservation module, for retaining the upper layer node in the case where the upper layer node is located at first sequence The second data transmission objective obtained in current detection cycle.

Optionally, the sorting module, comprising:

Submodule is determined, for determining the size relation of the specified data transmission objective in each upper layer node；Institute State the specified data transmission objective in upper layer node are as follows: the second data transmission objective of the weighted processing or the institute retained State the second data transmission objective；

Sorting sub-module, for being ranked up to each upper layer node according to identified size relation.

Optionally, the weighting processing module, is used for:

Wherein, the 4th default expression formula are as follows:

The third aspect, the embodiment of the invention provides a kind of server, including processor and machine readable storage medium, institutes It states machine readable storage medium and is stored with the machine-executable instruction that can be executed by the processor, the processor executes institute Machine-executable instruction is stated to realize the method and step of the data capture method of above-mentioned first aspect offer.

Fourth aspect, the embodiment of the invention provides a kind of computer readable storage medium, the computer-readable storage Dielectric memory contains computer program, when the computer program is executed by processor, realizes the number that above-mentioned first aspect provides According to the method and step of acquisition methods.

5th aspect, the embodiment of the invention also provides a kind of computer program products comprising instruction, when it is being calculated When being run on machine, so that computer executes the method and step for the data capture method that above-mentioned first aspect provides.

6th aspect, the embodiment of the invention also provides a kind of computer programs, when run on a computer, so that Computer executes the method and step for the data capture method that above-mentioned first aspect provides.

A kind of data capture method, data acquisition facility, server and storage medium provided in an embodiment of the present invention determine After the first data transmission index of each upper layer node of current detection cycle, by being pre-processed to first data transmission index And disposal of stability, the second data transmission objective is obtained, since the second data transmission objective is compared to first data transmission index More difficult fluctuation, therefore, the position sequence of each upper layer node is more stable after being ranked up every time to each upper layer node, to increase A possibility that fringing node obtains required data from same upper layer node avoids after switching because of upper layer node, by being switched Upper layer node in it is not stored needed for data so that upper layer node need further again to a node layer thereon request needed for data, Then the case where being successively back to fringe node again generation reduces data obtaining time to reduce data acquisition process, reduces Data acquisition cost.Certainly, it implements any of the products of the present invention or method must be not necessarily required to reach above-described institute simultaneously There is advantage.

Detailed description of the invention

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.

Fig. 1 is the simplified schematic diagram of CDN network framework in the related technology；

Fig. 2 is a kind of flow diagram of data capture method provided in an embodiment of the present invention；

Fig. 3 is a kind of flow diagram of step S102 in the embodiment of the present invention；

Fig. 4 is another flow diagram of data capture method provided in an embodiment of the present invention；

Fig. 5 is a kind of flow diagram of step S206 in data capture method provided in an embodiment of the present invention；

Fig. 6 is a kind of structural schematic diagram of data acquisition facility provided in an embodiment of the present invention；

Fig. 7 is a kind of structural schematic diagram of processing module in the embodiment of the present invention；

Fig. 8 is another structural schematic diagram of data acquisition facility provided in an embodiment of the present invention；

Fig. 9 is a kind of structural schematic diagram of sorting module in the embodiment of the present invention；

Figure 10 is a kind of structural schematic diagram of server provided in an embodiment of the present invention.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.

CDN in the related technology, it will usually configure multiple available upper layer node IP for fringe node, that is, fringe node can To select a data transmission objective best from multiple upper layer nodes, and data are obtained from selected upper layer node. Fringe node can be saved periodically by the locator of itself to each upper layer when determining the transmission quality of each upper layer node Point sends probe requests thereby, and each upper layer node returns to a default file after receiving probe requests thereby, which is typically sized to Tens KB (kilobytes)~a few MB (Mbytes).Upper layer node based on each upper layer node return default file speed of download, under The information such as time are carried, determine the data transmission objective of each upper layer node, and be ranked up to each upper layer node, Cong Weixu's first Data are obtained in upper layer node.In next detection cycle, since the transmission quality of each upper layer node changes, each upper layer Position sequence after node rearrangement also changes therewith.

That is, in the related technology when each detection cycle sorts to each upper layer node, the position sequence of each upper layer node Fluctuation is frequent, in fact it could happen that all different situation of the upper layer node of position sequence first after every minor sort selects fringe node every time Upper layer node it is all different, it will once data needed for not stored in the upper layer node for causing the fringe node to select, then upper layer Node needs data needed for further requesting again to a node layer thereon, is then back to the fringe node in layer again, causes Data acquisition process increases, and data obtaining time is caused to extend.

For example, in a preceding detection cycle for current detection cycle, upper layer node A is located at first sequence, upper layer node B In second sequence, and data needed for fringe node are stored in upper layer node A, in upper layer node B needed for not stored fringe node Data, then fringe node can directly obtain required data from the upper layer node A for being located at first sequence；And in current detection week Phase, since position sequence fluctuates, upper layer node B drains into first sequence, and upper layer node A drains into second sequence, when fringe node is from being located at When obtaining data in the upper layer node B of first sequence, the data as needed for not stored fringe node in upper layer node B, then upper layer Node B needs data (i.e. cold file download) needed for further requesting to a node layer thereon, data needed for then successively returning.

Also, CDN in the related technology, the upper layer node big for data transmission objective difference can distinguish, but It is that the upper layer node small for data transmission objective difference is then difficult to differentiate between, needs through manual method, manually separate data Upper layer node similar in transmission objective causes CND provider operation cost to increase.

Embodiment of the method 1

Have in view of that, as shown in Fig. 2, can be applied to CDN net the embodiment of the invention provides a kind of data capture method Fringe node in network, naturally it is also possible to which, applied to the other kinds of equipment with server capability, this method may include Following steps:

S101 determines the first data transmission index of each upper layer node of current detection cycle.

In the embodiment of the present invention, it is illustrated by taking fringe node as an example.Fringe node can configure multiple available upper layer sections Point IP, when user needs to obtain data, if the required data that fringe node does not cache, fringe node can be to upper layer Data needed for node obtains.Wherein, fringe node can refer to the cache server near user, be usually deployed in entire The network edge of CDN, for obtaining data from upper layer node and caching；Upper layer node can refer to source server, that is, be stored with The data center that the server of source data, usually layering are deployed in entire CDN, for storing data.

Fringe node can detect the original data transmissions index of each upper layer node, for example, passing through the detection itself installed Probe requests thereby is sent to routines periodically to each upper layer node, each upper layer node returns to a default text after receiving probe requests thereby Part, the default file are typically sized to the KB of several KB~tens, analyze for the transmission speed to link.Upper layer node is based on Each upper layer node returns to the information such as speed of download, the download time of default file, determines the original data transmissions of each upper layer node Index, i.e., described first data transmission index in the embodiment of the present invention.

Above-mentioned detection cycle can be illustratively 10 seconds, 1 minute, 3 minutes etc..For 1 minute, fringe node can To send probe requests thereby to each upper layer node every 1 minute, to determine the first number of primary each upper layer node every one minute According to transmission objective.Certainly, server admin personnel can need the above-mentioned detection cycle of flexible setting, this hair according to actual business Details are not described herein for bright embodiment.Current detection cycle can be current signified one of detection in multiple detection cycles Period.

Wherein, first data transmission index may include: the transmission time of data.It is readily appreciated that, if the transmission of data Time is small, then data needed for fringe node can spend less time acquisition, therefore data transmission objective is high, this is for improving User experience plays positive effect.Certainly, in addition to the transmission time of data, other transmission times with data change and change Index, can be used for evaluation first data transmission index, for example, the transmission speed of data.This is because, working as the biography of data When the defeated time shortens, the transmission speed of data is increased with it.Certainly, except for example shown in addition to implementation, realize The mode of this feature belongs to the protection scope of the embodiment of the present invention.

S102, for each upper layer node in each upper layer node, to the first data transmission index of upper layer node into Row pretreatment and disposal of stability, obtain the second data transmission objective of upper layer node.

In the embodiment of the present invention, in the original data transmissions index for obtaining each upper layer node, i.e., each upper layer node After first data transmission index, pretreatment and disposal of stability can be carried out to first data transmission index, to make through handling The new data transmission objective obtained afterwards, i.e. the second data transmission objective have smaller fluctuation, that is, having higher steady It is qualitative.Illustratively, above-mentioned pretreatment may include the normalized and accumulation process two parts of data.

S103, the size of the second data transmission objective based on each upper layer node, is ranked up each upper layer node.

In step of the embodiment of the present invention, after obtaining the second data transmission objective of each upper layer node, each upper layer section Second data transmission objective of point it is of different sizes, therefore can be according to the size order of the second data transmission objective, to each Upper layer node is ranked up.

Illustratively, if the second data transmission objective of obtained upper layer node A is a, the of obtained upper layer node B Two data transmission objectives are b, and the second data transmission objective of obtained upper layer node C is c, and a > b > c, then in above three It can be with after node layer sequence are as follows: upper layer node A- upper layer node B- upper layer node C, wherein upper layer node A is located at first sequence, Upper layer node B is located at second sequence, and upper layer node C is located at third position sequence.

S104, into each upper layer node after sequence, the upper layer node of position sequence first sends data acquisition request, so that The upper layer node of position sequence first returns to request data corresponding with data acquisition request.

After sorted, fringe node can determine the upper layer node of a sequence first to each upper layer node, and then can be to this Upper layer node sends data acquisition request.It, can be to fringe node after the upper layer node of position sequence first receives data acquisition request Return to request data corresponding with data acquisition request.It analyzes on the whole, due to the stability phase of the second data transmission objective It is higher than the stability in first data transmission index, therefore after being ranked up every time to each upper layer node, each upper layer section The position sequence of point is more stable, the more difficult change of the upper layer node of first sequence, and fringe node obtains institute from same upper layer node The probability of data is needed to increase, to reduce the generation of cold file download conditions.

As a kind of optional embodiment of the embodiment of the present invention, above-mentioned steps S102, as shown in figure 3, specifically including:

S1021 is normalized the first data transmission index of upper layer node, obtains upper layer node and visit currently Survey the third data transmission objective in period.

It, can be to identified first data after fringe node determines the first data transmission index of each upper layer node Transmission objective is normalized, that is, nondimensional expression formula will be converted to after the expression formula for having dimension transformation, to simplify Processing, normalized can be a part in above-mentioned pretreatment.

Illustratively, when first data transmission index is the transmission time of data, it is the numerical value for having dimension, can By normalized, the first data transmission index for having dimension is converted to nondimensional third data transmission objective.

Specifically, the first data transmission index of each upper layer node can equally be had to the number of identical dimension divided by one Value, to eliminate dimension, obtains a dimensionless number.Certainly, for the first data transmission index lifted in embodiment Other examples, the method that can use above-mentioned elimination dimension, obtain the third data transmission objective of each upper layer node.

As a kind of optional embodiment of the embodiment of the present invention, it can be based on the first default expression formula, to upper layer node First data transmission index be normalized；

Wherein, the first default expression formula are as follows:

In formula,Indicate the third data transmission objective of i-th of upper layer node in each upper layer node；t_iIndicate the The first data transmission index of i upper layer node；t_minIndicate that the first data transmission index in all upper layer nodes is the smallest by One data transmission objective.

S1022 carries out accumulation process to all third data transmission objectives of the upper layer node in predetermined time period, obtains To the 4th data transmission objective of upper layer node；Wherein, predetermined time period includes current detection cycle and the detection of multiple history Period.

According to foregoing teachings it is found that fringe node can periodically detect each upper layer node, that is to say, that every The corresponding first data transmission index of each upper layer node can be determined after secondary detection, and then obtains third data transmission objective. So, in a predetermined time period, then multiple third data transmission objectives of available each upper layer node.

Therefore, all third data transmission objectives of each upper layer node in predetermined time period can be carried out with cumulative place Reason, to obtain the 4th data transmission objective of each upper layer node, accumulation process can be above-mentioned pretreated a part.

For example, detection cycle is 1 minute, predetermined time period is 90 minutes, for upper layer node A, then available 90 A third data transmission objective, can be with during accumulation process by this 90 third data transmission objective accumulation process Using each third data transmission objective as the index of parameter preset, to obtain a 4th data transmission objective.

By above-mentioned example it is found that predetermined time period may include a current detection cycle and multiple history detection week Phase, for example, having 1 current detection cycle and 89 history detection cycles in predetermined time period 90 minutes.Certainly, it services Device administrative staff can need the above-mentioned predetermined time period of flexible setting according to actual business, for example, it is also possible to be set as 60 Minute, 120 minutes, details are not described herein for the embodiment of the present invention.

By to third data transmission objective carry out accumulation process, the 4th data transmission objective being calculated and it is default when Between all third data transmission objectives in length be related, therefore, the 4th data transmission objective is passed compared to the first data The fluctuation tendency of defeated index is smaller.

As a kind of optional embodiment of the embodiment of the present invention, each upper layer can be saved by the second default expression formula All third data transmission objectives of the point in predetermined time period carry out accumulation process；

Wherein, the second default expression formula are as follows:

In formula,Indicate the 4th data transmission objective of i-th of upper layer node in each upper layer node；Period table Show predetermined time period, for example, 90min；J indicates the detection cycle number that predetermined time period includes, for example, 90；α is indicated First parameter preset, for example, 1.7, Parameter adjustable；Indicate the third data transmission objective of i-th of upper layer node；I is Integer greater than 0.

S1023 carries out disposal of stability to the 4th data transmission objective of upper layer node, obtains the second number of upper layer node According to transmission objective.

In the embodiment of the present invention, after obtaining the 4th data transmission objective of each upper layer node, it can continue to save each upper layer 4th data transmission objective of point carries out disposal of stability, to further decrease the fluctuation range of numerical value.In disposal of stability mistake Cheng Zhong, can be using the 4th data transmission objective as the antilog of default logarithmic function, to obtain the second data transmission objective.

After disposal of stability, the size model of the magnitude range of the second data transmission objective than the 4th data transmission objective Enclose smaller, therefore compared to the 4th data transmission objective, the second data transmission objective has smaller fluctuation, that is, second Data transmission objective has higher stability.

As a kind of optional embodiment of the embodiment of the present invention, expression formula can be preset by third, each upper layer is saved 4th data transmission objective of point carries out disposal of stability；

Wherein, third presets expression formula are as follows:

In formula,Indicate the second data transmission objective of i-th of upper layer node in each upper layer node；X indicates pre- It can be 2, natural constant e or 10 if the truth of a matter；Indicate the 4th data transmission objective of i-th of upper layer node；I is Integer greater than 0.

A kind of data capture method provided in an embodiment of the present invention determines the first of each upper layer node of current detection cycle After data transmission objective, by carrying out pretreatment and disposal of stability to first data transmission index, the transmission of the second data is obtained Index, since the second data transmission objective is compared to the more difficult fluctuation of first data transmission index, every time on each The position sequence that node layer is ranked up rear each upper layer node is more stable, obtains institute from same upper layer node to increase fringe node A possibility that needing data, avoids after switching because of upper layer node, by data needed for not stored in the upper layer node that is switched, so that The case where upper layer node needs data needed for further requesting again to a node layer thereon, is then successively back to fringe node again hair It is raw, to reduce data acquisition process, data obtaining time is reduced, reduces data acquisition cost.

Embodiment of the method 2

As shown in figure 4, can be applied in CDN network the embodiment of the invention also provides a kind of data capture method Fringe node, naturally it is also possible to which, applied to the other kinds of equipment with server capability, this method may include following step It is rapid:

S201 determines the first data transmission index of each upper layer node of current detection cycle.

Fringe node can detect the data transmission objective of each upper layer node, for example, passing through the locator itself installed Probe requests thereby periodically is sent to each upper layer node, each upper layer node returns to a default file after receiving probe requests thereby, should Default file is typically sized to tens KB~a few MB, for example, 2MB, is analyzed for the transmission speed to link.Upper layer node The information such as speed of download, the download time of default file are returned based on each upper layer node, determine the data transmission of each upper layer node Index, i.e., described first data transmission index in the embodiment of the present invention.

S202, for each upper layer node in each upper layer node, to the first data transmission index of upper layer node into Row pretreatment and disposal of stability, obtain the second data transmission objective of upper layer node.

S203 judges the sequence of upper layer node each upper layer node in a preceding detection cycle for current detection cycle In, if it is located at first sequence.

The embodiment of the present invention can first judge upper layer node in current detection cycle before to the sequence of each upper layer node In a preceding detection cycle in the sequence of each upper layer node, if be first sequence, and difference can be done according to judging result Processing.

S204 obtains upper layer node in the case where upper layer node is not located at first sequence in current detection cycle The second data transmission objective be weighted processing, obtain weighting treated the second data transmission objective.

For being not located at the upper layer node of first sequence, can work as in a preceding detection cycle for current detection cycle In the calculating in preceding period, increased in a preceding detection cycle by weighting processing, be not located at the upper layer node of first sequence with The difference of the second data transmission objective between the upper layer node of first sequence.For example, the second data transmission objective can be made to multiply It is greater than 1 weighting coefficient with one, which may be considered a penalty factor.

It, can be by the 4th default expression formula, to current detection as a kind of optional embodiment of the embodiment of the present invention In a preceding detection cycle in period in the sequence of each upper layer node, it is not located at the second data of the upper layer node of first sequence Transmission objective is weighted processing；

4th default expression formula are as follows:

In formula, m indicates weighting coefficient；Count_iIndicate that i-th of upper layer node is long in preset time in each upper layer node It is located at the number of first sequence in degree；Count_*It indicates in a preceding detection cycle for current detection cycle, is located at first sequence Upper layer node predetermined time period internal sort be first sequence total degree；β indicates the second parameter preset, for example, 1.4, Parameter adjustable；γ indicates third parameter preset, for example, 1.4, Parameter adjustable, wherein as β and γ become larger, Ranking Stability liter Height, susceptibility reduce；I is the integer greater than 0.

S205 retains upper layer node and obtains in current detection cycle in the case where upper layer node is located at first sequence The second data transmission objective.

For that positioned at the upper layer node of first sequence, then can retain in a preceding detection cycle for current detection cycle The upper layer node second data transmission objective obtained in current detection cycle.By aforesaid operations, if a upper layer section Point is located at first sequence in a preceding detection cycle for current detection cycle, then upper layer node position in current detection cycle It is bigger in a possibility that first sequence will be than other upper layer nodes a possibility that, just it is located at the originally that is, further improving Stability of the upper layer node of one sequence in subsequent probe cyclic scheduling.

S206, the size of the second data transmission objective based on each upper layer node, is ranked up each upper layer node.

Illustratively, if the second data transmission objective of obtained upper layer node A is a, the of obtained upper layer node B Two data transmission objectives are b, and the second data transmission objective of obtained upper layer node C is c, and c > b > a, when the smaller expression of numerical value When detection quality is better, then to can be with after the sequence of above three upper layer node are as follows: upper layer node A- upper layer node B- upper layer node C, wherein upper layer node A is located at first sequence, and upper layer node B is located at second sequence, and upper layer node C is located at third position sequence.When So, when the bigger expression detection quality of a, b, c numerical value is better, ordering principle is identical, and details are not described herein.

As a kind of optional embodiment of the embodiment of the present invention, as shown in figure 5, above-mentioned steps S206 specifically can wrap It includes:

S2061, the size relation for determining specified data transmission objective in each upper layer node.

In the embodiment of the present invention, specified data transmission objective in upper layer node can be with are as follows: the second number of weighted processing According to transmission objective or the second data transmission objective retained, the second data transmission objective of multiple weighted processing is obtained, with And after a second data transmission objective retained, thus may determine that the size relation between them.

S2062, according to identified size relation, each upper layer node is ranked up.

It, can be according to identified since each second data transmission objective can correspond to a upper layer node Size relation between two data transmission objectives, is ranked up each upper layer node.

S207, into each upper layer node after sequence, the upper layer node of position sequence first sends data acquisition request, so that The upper layer node of position sequence first returns to request data corresponding with data acquisition request.

After sorted, fringe node can determine the upper layer node of a sequence first to each upper layer node, and then can be to this Upper layer node sends data acquisition request.It, can be to fringe node after the upper layer node of position sequence first receives data acquisition request Return to request data corresponding with data acquisition request.It analyzes on the whole, due to the stability phase of the second data transmission objective It is higher than the stability in first data transmission index, therefore after being ranked up every time to each upper layer node, each upper layer node Position sequence it is more stable, the more difficult change of the upper layer node of first sequence, fringe node from same upper layer node acquisition needed for The probability of data increases, to reduce the generation of cold file download conditions.

A kind of data capture method provided in an embodiment of the present invention judges that upper layer node is primary before current detection cycle After whether being located at first sequence in the sequence of each upper layer node in detection cycle, the feelings of first sequence are not located in upper layer node Under condition, then the upper layer node second data transmission objective obtained in current detection cycle is weighted processing, Ke Yizeng It is added in a preceding detection cycle, is not located at the upper layer node of first sequence and the second number between the upper layer node of first sequence According to the difference of transmission objective；In the case where upper layer node is located at first sequence, retain the upper layer node in current detection cycle Obtained in the second data transmission objective, further improve originally just positioned at first sequence upper layer node subsequent probe week Stability when phase sorts, to further increase a possibility that fringe node obtains required data from same upper layer node.

Installation practice 1

Corresponding to above method embodiment, embodiment that the embodiment of the present invention also provides corresponding devices.

As shown in fig. 6, the embodiment of the invention provides a kind of data acquisition facilities, comprising:

Determining module 301, the first data transmission index of each upper layer node for determining current detection cycle.

Processing module 302, each upper layer node for being directed in each upper layer node, to the first data of upper layer node Transmission objective carries out pretreatment and disposal of stability, obtains the second data transmission objective of upper layer node.

Sorting module 303 saves each upper layer for the size of the second data transmission objective based on each upper layer node Point is ranked up.

Sending module 304, the upper layer node for the position sequence first into each upper layer node after sequence send data and obtain Request is taken, so that the upper layer node of position sequence first returns to request data corresponding with data acquisition request.

Wherein, as shown in fig. 7, above-mentioned processing module 302, comprising:

Normalized submodule 3021 is normalized for the first data transmission index to upper layer node, Upper layer node is obtained in the third data transmission objective of current detection cycle.

Accumulation process submodule 3022, for referring to all third data transmission of the upper layer node in predetermined time period Mark carries out accumulation process, obtains the 4th data transmission objective of upper layer node；Wherein, predetermined time period includes current detection week Phase and multiple history detection cycles.

Disposal of stability submodule 3023 carries out disposal of stability for the 4th data transmission objective to upper layer node, Obtain the second data transmission objective of upper layer node.

Wherein, normalized submodule 3021, is used for:

Wherein, the first default expression formula are as follows:

In formula,Indicate the third data transmission objective of i-th of upper layer node in each upper layer node；t_iIndicate the The first data transmission index of i upper layer node；t_minIndicate that the first data transmission index in all upper layer nodes is the smallest by One data transmission objective；I is the integer greater than 0.

Wherein, accumulation process submodule 3022, is specifically used for:

By the second default expression formula, to all third data transmission objectives of the upper layer node in predetermined time period into Row accumulation process；

Wherein, the second default expression formula are as follows:

In formula,Indicate the 4th data transmission objective of i-th of upper layer node in each upper layer node；Period table Show predetermined time period；J indicates the detection cycle number that predetermined time period includes；α indicates the first parameter preset；Table Show the third data transmission objective of i-th of upper layer node；I is the integer greater than 0.

Wherein, disposal of stability submodule 3023, is specifically used for:

Third presets expression formula are as follows:

A kind of data acquisition facility provided in an embodiment of the present invention determines the first of each upper layer node of current detection cycle After data transmission objective, by carrying out pretreatment and disposal of stability to first data transmission index, the transmission of the second data is obtained Index, since the second data transmission objective is compared to the more difficult fluctuation of first data transmission index, every time on each The position sequence that node layer is ranked up rear each upper layer node is more stable, obtains institute from same upper layer node to increase fringe node A possibility that needing data, avoids after switching because of upper layer node, by data needed for not stored in the upper layer node that is switched, so that The case where upper layer node needs data needed for further requesting again to a node layer thereon, is then successively back to fringe node again hair It is raw, to reduce data acquisition process, data obtaining time is reduced, reduces data acquisition cost.

Installation practice 2

The embodiment of the invention also provides a kind of data acquisition facility, on the basis of example structure shown in Fig. 6, such as scheme Shown in 8, can also include:

Judgment module 401, for judging upper layer node each upper layer in a preceding detection cycle for current detection cycle In the sequence of node, if be located at first sequence；

Processing module 402 is weighted, in the case where upper layer node is not located at first sequence, by upper layer node current Second data transmission objective obtained in detection cycle is weighted processing, obtains weighting treated the second data transmission and refers to Mark；

Reservation module 403 is detected in the case where upper layer node is located at first sequence, retaining upper layer node currently Second data transmission objective obtained in period.

Wherein, above-mentioned sorting module 303, as shown in Figure 9, comprising:

Submodule 3031 is determined, for determining the size relation of the specified data transmission objective in each upper layer node；On Specified data transmission objective in node layer are as follows: the second data transmission objective of weighted processing or the second data retained pass Defeated index.

Sorting sub-module 3052, for being ranked up to each upper layer node according to identified size relation.

Wherein, above-mentioned weighting processing module 402, is used for:

Sequence by the 4th default expression formula, to each upper layer node in a preceding detection cycle for current detection cycle In, the second data transmission objective for being not located at the upper layer node of first sequence is weighted processing；

Wherein, the 4th default expression formula are as follows:

In formula, m indicates weighting coefficient；Count_iIndicate that i-th of upper layer node is long in preset time in each upper layer node It is located at the number of first sequence in degree；Count_*It indicates in a preceding detection cycle for current detection cycle, is located at first sequence Upper layer node predetermined time period internal sort be first sequence total degree；β indicates the second parameter preset；γ indicates third Parameter preset；I is the integer greater than 0.

A kind of data acquisition facility provided in an embodiment of the present invention judges that upper layer node is primary before current detection cycle After whether being located at first sequence in the sequence of each upper layer node in detection cycle, the feelings of first sequence are not located in upper layer node Under condition, then the upper layer node second data transmission objective obtained in current detection cycle is weighted processing, Ke Yizeng It is added in a preceding detection cycle, is not located at the upper layer node of first sequence and the second number between the upper layer node of first sequence According to the difference of transmission objective；In the case where upper layer node is located at first sequence, retain the upper layer node in current detection cycle Obtained in the second data transmission objective, further improve originally just positioned at first sequence upper layer node subsequent probe week Stability when phase sorts, to further increase a possibility that fringe node obtains required data from same upper layer node.

The embodiment of the invention also provides a kind of servers, are specifically as follows server, as shown in Figure 10, the equipment 500 Including processor 501 and machine readable storage medium 502, machine readable storage medium is stored with the machine that can be executed by processor Device executable instruction, processor executes the step of machine-executable instruction realizes the data capture method of any of the above-described, optional , following steps may be implemented:

For each upper layer node in each upper layer node, the first data transmission index of upper layer node is located in advance Reason and disposal of stability, obtain the second data transmission objective of upper layer node；

The size of the second data transmission objective based on each upper layer node, is ranked up each upper layer node；

Into each upper layer node after sequence, the upper layer node of position sequence first sends data acquisition request, so that position sequence the One upper layer node returns to request data corresponding with data acquisition request.

Machine readable storage medium may include random access memory (Random Access Memory, abbreviation RAM), It also may include nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.Optionally, Memory can also be that at least one is located remotely from the storage device of aforementioned processor.

Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, Abbreviation CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be digital signal processor (Digital Signal Processing, abbreviation DSP), specific integrated circuit (Application Specific Integrated Circuit, abbreviation ASIC), field programmable gate array (Field-Programmable Gate Array, Abbreviation FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware components.

Server provided in an embodiment of the present invention determines that the first data transmission of each upper layer node of current detection cycle refers to After mark, by carrying out pretreatment and disposal of stability to first data transmission index, the second data transmission objective is obtained, due to the Two data transmission objectives are compared to the more difficult fluctuation of first data transmission index, therefore, carrying out every time to each upper layer node The position sequence of each upper layer node is more stable after sequence, thus increase fringe node from same upper layer node obtain needed for data can Can property, avoid after switching because of upper layer node, by data needed for not stored in the upper layer node that is switched, so that upper layer node needs The case where further requesting required data to a node layer thereon again, being then successively back to fringe node again generation, to subtract Few data acquisition process, reduces data obtaining time, reduces data acquisition cost.

The embodiment of the invention also provides a kind of computer readable storage medium, it is stored in computer readable storage medium Computer program, can the step of data capture method to execute any of the above-described when computer program is executed by processor Choosing, following steps can be executed:

Computer readable storage medium provided in an embodiment of the present invention determines the of each upper layer node of current detection cycle After one data transmission objective, by carrying out pretreatment and disposal of stability to first data transmission index, the second data biography is obtained Defeated index, since the second data transmission objective is compared to the more difficult fluctuation of first data transmission index, every time to each The position sequence that upper layer node is ranked up rear each upper layer node is more stable, obtains to increase fringe node from same upper layer node A possibility that required data, avoids after switching because of upper layer node, by data needed for not stored in the upper layer node that is switched, makes Obtain the case where upper layer node needs data needed for further requesting again to a node layer thereon, is then successively back to fringe node again Occur, to reduce data acquisition process, reduces data obtaining time, reduce data acquisition cost.

The embodiment of the invention also provides a kind of computer program products comprising instruction, when it runs on computers When, so that the step of computer executes the data capture method of any of the above-described, optionally, can execute following steps:

Computer program product provided in an embodiment of the present invention comprising instruction determines each upper layer section of current detection cycle After the first data transmission index of point, by carrying out pretreatment and disposal of stability to first data transmission index, second is obtained Data transmission objective, since the second data transmission objective is compared to the more difficult fluctuation of first data transmission index, every The secondary position sequence for being ranked up rear each upper layer node to each upper layer node is more stable, saves to increase fringe node from same upper layer A possibility that data needed for point obtains, avoids after switching because of upper layer node, needed for not stored in the upper layer node that is switched Then data are successively back to edge section so that upper layer node needs data needed for further requesting again to a node layer thereon again The case where putting reduces data obtaining time to reduce data acquisition process, reduces data acquisition cost.

The embodiment of the invention also provides a kind of computer programs, when run on a computer, so that computer is held The step of data capture method of row any of the above-described, can optionally execute following steps:

Computer program provided in an embodiment of the present invention comprising instruction, determines each upper layer node of current detection cycle After first data transmission index, by carrying out pretreatment and disposal of stability to first data transmission index, the second data are obtained Transmission objective, since the second data transmission objective is compared to the more difficult fluctuation of first data transmission index, right every time The position sequence that each upper layer node is ranked up rear each upper layer node is more stable, obtains to increase fringe node from same upper layer node A possibility that taking required data avoids after switching because of upper layer node, by data needed for not stored in the upper layer node that is switched, So that upper layer node needs data needed for further requesting again to a node layer thereon, it is then successively back to the feelings of fringe node again Condition occurs, to reduce data acquisition process, reduces data obtaining time, reduces data acquisition cost.

For device/server/storage medium embodiment, since it is substantially similar to the method embodiment, so retouching That states is fairly simple, and the relevent part can refer to the partial explaination of embodiments of method.

It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including element.

Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method Part explanation.

The above is merely preferred embodiments of the present invention, it is not intended to limit the scope of the present invention.It is all in this hair Any modification, equivalent replacement, improvement and so within bright spirit and principle, are included within the scope of protection of the present invention.

Claims

1. a kind of data capture method, which is characterized in that the described method includes:

For each upper layer node in each upper layer node, the first data transmission index of the upper layer node is carried out Pretreatment and disposal of stability, obtain the second data transmission objective of the upper layer node；

The size of the second data transmission objective based on each upper layer node, arranges each upper layer node Sequence；

Into each upper layer node after sequence, the upper layer node of position sequence first sends data acquisition request, so that position sequence the One upper layer node returns to request data corresponding with the data acquisition request.

2. the method according to claim 1, wherein the first data transmission to the upper layer node Index carries out pretreatment and disposal of stability, the second data transmission objective for obtaining the upper layer node include:

The first data transmission index of the upper layer node is normalized, obtains the upper layer node current The third data transmission objective of detection cycle；

Accumulation process is carried out to all third data transmission objectives of the upper layer node in predetermined time period, is obtained 4th data transmission objective of the upper layer node；Wherein, the predetermined time period includes current detection cycle and multiple goes through History detection cycle；

Disposal of stability is carried out to the 4th data transmission objective of the upper layer node, obtains the second of the upper layer node Data transmission objective.

3. according to the method described in claim 2, it is characterized in that, the first data transmission to the upper layer node Index is normalized, and obtains the upper layer node the third data transmission objective of current detection cycle the step of, packet It includes:

Wherein, the described first default expression formula are as follows:

In formula,Indicate the third data transmission objective of i-th of upper layer node in each upper layer node；t_iIndicate described i-th The first data transmission index of a upper layer node；t_minIndicate that the first data transmission index in all upper layer nodes is the smallest by One data transmission objective；I is the integer greater than 0.

4. according to the method described in claim 2, it is characterized in that, it is described to the upper layer node in predetermined time period All third data transmission objectives carry out accumulation process, obtain the step of the 4th data transmission objective of the upper layer node Suddenly, comprising:

Wherein, the described second default expression formula are as follows:

In formula,Indicate the 4th data transmission objective of i-th of upper layer node in each upper layer node；Period indicates pre- If time span；J indicates the detection cycle number that the predetermined time period includes；≤ indicate the first parameter preset； Indicate the third data transmission objective of i-th of upper layer node；I is the integer greater than 0.

5. according to the method described in claim 2, it is characterized in that, the 4th data to the upper layer node are transmitted The step of index carries out disposal of stability, obtains the second data transmission objective of the upper layer node, comprising:

Wherein, the third presets expression formula are as follows:

In formula,Indicate the second data transmission objective of i-th of upper layer node in each upper layer node；X indicates default bottom Number；Indicate the 4th data transmission objective of i-th of upper layer node；I is the integer greater than 0.

6. according to the method described in claim 2, it is characterized in that, being passed in the 4th data to the upper layer node Defeated index carries out disposal of stability, after obtaining the second data transmission objective of the upper layer node, comprising:

Judge the upper layer node in a preceding detection cycle for the current detection cycle in the sequence of each upper layer node, Whether first sequence is located at；

In the case where the upper layer node is not located at first sequence, by the upper layer node in current detection cycle To the second data transmission objective be weighted processing, obtain weighting treated the second data transmission objective；

In the case where the upper layer node is located at first sequence, retains the upper layer node and obtained in current detection cycle The the second data transmission objective arrived.

7. according to the method described in claim 6, it is characterized in that, second number based on each upper layer node According to the size of transmission objective, the step of being ranked up to each upper layer node, comprising:

Determine the size relation of the specified data transmission objective in each upper layer node；Specified number in the upper layer node According to transmission objective are as follows: weighting treated the second data transmission objective or the second data transmission objective retained；

According to identified size relation, each upper layer node is ranked up.

8. according to the method described in claim 6, it is characterized in that, described obtain the upper layer node in current detection cycle To the second data transmission objective be weighted processing, obtain weighting treated the second data transmission objective the step of, Include:

Sequence by the 4th default expression formula, to each upper layer node in a preceding detection cycle for the current detection cycle In, the second data transmission objective for being not located at the upper layer node of first sequence is weighted processing；

The 4th default expression formula are as follows:

In formula, m indicates weighting coefficient；Count_iIndicate that i-th of upper layer node is in the predetermined time period in each upper layer node Inside it is located at the number of first sequence；Count_*It indicates in a preceding detection cycle for the current detection cycle, is located at first The upper layer node of sequence is in the total degree that the predetermined time period internal sort is first sequence；β indicates the second parameter preset；γ table Show third parameter preset；I is the integer greater than 0.

9. a kind of data acquisition facility, which is characterized in that described device includes:

Processing module, for being counted to described the first of the upper layer node for each upper layer node in each upper layer node Pretreatment and disposal of stability are carried out according to transmission objective, obtains the second data transmission objective of the upper layer node；

Sorting module, for the size of the second data transmission objective based on each upper layer node, to each described Upper layer node is ranked up；

Sending module, the upper layer node for the position sequence first into each upper layer node after sequence send data acquisition and ask It asks, so that the upper layer node of position sequence first returns to request data corresponding with the data acquisition request.

10. device according to claim 9, which is characterized in that the processing module, comprising:

Normalized submodule is normalized for the first data transmission index to the upper layer node, The upper layer node is obtained in the third data transmission objective of current detection cycle；

Accumulation process submodule, for referring to all third data transmission of the upper layer node in predetermined time period Mark carries out accumulation process, obtains the 4th data transmission objective of the upper layer node；Wherein, the predetermined time period includes working as Preceding detection cycle and multiple history detection cycles；

Disposal of stability submodule carries out disposal of stability for the 4th data transmission objective to the upper layer node, Obtain the second data transmission objective of the upper layer node.

11. a kind of server, which is characterized in that including processor and machine readable storage medium, the machine readable storage is situated between Matter is stored with the machine-executable instruction that can be executed by the processor, and the processor executes the machine-executable instruction To realize the described in any item method and steps of claim 1-8.

12. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium Program realizes claim 1-8 described in any item method and steps when the computer program is executed by processor.