CN117834730B - CDN (content delivery network) resource preheating method, device, equipment and storage medium - Google Patents
CDN (content delivery network) resource preheating method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117834730B CN117834730B CN202410239111.1A CN202410239111A CN117834730B CN 117834730 B CN117834730 B CN 117834730B CN 202410239111 A CN202410239111 A CN 202410239111A CN 117834730 B CN117834730 B CN 117834730B
- Authority
- CN
- China
- Prior art keywords
- url
- node
- preheating
- edge node
- resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012384 transportation and delivery Methods 0.000 title abstract description 5
- 230000008569 process Effects 0.000 claims description 18
- 238000009825 accumulation Methods 0.000 claims description 10
- 230000036541 health Effects 0.000 claims description 10
- 230000004931 aggregating effect Effects 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 5
- 238000004088 simulation Methods 0.000 claims description 5
- 238000010792 warming Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims 2
- 230000006872 improvement Effects 0.000 abstract description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010438 heat treatment Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for preheating resources of a CDN (content delivery network), which relate to the technical computer field, and comprise the following steps: acquiring a resource access log; accumulating the resource access log, and determining each hotspot url according to the accumulated result; analyzing the states of all edge nodes in the CDN to obtain node state list information; and distributing any hot point url to a corresponding edge node based on the node state list information so as to preheat the content of the hot point url in the edge node. According to the method and the system, the hot spot resources are predicted in advance according to the indexes such as the access duty ratio improvement of the resources, so that the triggering of completing the preheating task by counting the request information of the resources is realized, the automatic preferential selection of the hot spot resource issuing nodes can be completed, the influence of the excessive load of the node resources on the service quality of the existing network is avoided, the service quality of clients can be improved rapidly, and the preheating resources are comprehensive.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for preheating resources in a CDN network.
Background
With the rapid growth and popularity of the internet, there is an increasing demand for a fast, stable, efficient network access experience. To meet this demand, content Delivery Network (CDN) technology has evolved. The CDN technology distributes the content to a plurality of nodes by deploying the nodes on the Internet, so that a user can acquire the required content nearby, and the access speed and the stability are improved. However, as the service including short video is gradually increased in the service proportion of the whole network, the problem that the audio and video contents are blocked frequently caused by the problems of network, resource and the like seriously affects the experience of netizens. How to improve the overall quality of service performance of the CDN is one of the important issues to be addressed. The preheating of the resource content involves preloading the content of the client source station in advance, improves the hit rate and success rate of the edge node, reduces the CDN source-returning ratio, reduces the link performance loss caused by the source-returning process, and has important significance for improving the performance and stability of the CDN.
At present, the preheating of the resource content is more focused on optimization under the framework of a passive preheating service mechanism of a user/client side, namely, the identification of the hot content is completely based on the expected judgment of the client, so that the improvement of indexes such as the service hit rate and the like of the hot content is excessively dependent on the preheating behavior of the client side to a certain extent.
Disclosure of Invention
Based on the foregoing, it is necessary to provide a method, an apparatus, a device and a storage medium for preheating resources in a CDN network, so as to solve the technical problem that the identification of the hot content is completely based on the expected judgment of the client, and thus the improvement of the indexes such as the service hit rate for the hot content is too dependent on the preheating behavior of the client to a certain extent.
The invention provides a resource preheating method of a CDN (content delivery network), which comprises the following steps:
Acquiring a resource access log;
Accumulating the resource access log, and determining each hotspot url according to the accumulated result;
Analyzing the states of all edge nodes in the CDN to obtain node state list information;
and distributing any hot url to a corresponding edge node based on the node state list information so as to preheat the content of the hot url in the edge node.
Optionally, according to the method for preheating resources of a CDN network provided by the present invention, the accumulated result includes a total number of requests or a total flow of requests of url;
the step of accumulating the resource access log, and determining each hotspot url according to the accumulated result, includes:
Based on the resource access log, periodically accumulating the total number of requests or the total flow of requests of each url;
And if the total number of the requests of the url is larger than a preset number threshold or the total flow of the requests is larger than a preset flow threshold, taking the url as the hot spot url.
Optionally, according to the method for preheating resources of a CDN network provided by the present invention, the analyzing the states of each edge node in the CDN network to obtain node state list information includes:
Acquiring service process state information, node hardware resource load state information and node service traffic load state information of any edge node in a preset period;
Scoring the health degree of any edge node based on service process state information, node hardware resource load state information and node service traffic load state information corresponding to any edge node to obtain a scoring result of any edge node;
and selecting and obtaining the node state list information based on a preset score threshold and a scoring result of any edge node.
Optionally, according to the method for preheating resources of a CDN network provided by the present invention, the distributing any one of the hot urls to the corresponding edge node based on the node state list information includes:
for any of the hot spots url:
selecting each target node with configuration association from each edge node based on the domain name information of the hotspot url;
and ordering all the target nodes based on the node state list information to obtain edge nodes corresponding to the hot point url, so as to distribute the hot point url to the corresponding edge nodes.
Optionally, according to the method for preheating resources of a CDN network provided by the present invention, the distributing the hotspot url to a corresponding edge node includes:
grouping and aggregating all hot point url belonging to the same edge node;
and sending the aggregated hot point url to the corresponding edge node in batches through multiple threads.
Optionally, according to the method for preheating resources of a CDN network provided by the present invention, the performing content preheating on the hotspot url in the edge node includes:
analyzing by using an asynchronous thread to obtain the hot point url;
a multithreaded simulation client initiates a request for accessing the hotspot url;
and when the nginx receives the request, forwarding the request to a proxy server, and initiating a resource request to a source station by the proxy server and caching the response content to finish the content preheating of the hot url.
Optionally, according to the method for preheating resources of a CDN network provided by the present invention, after the content of the hotspot url is preheated in the edge node, the method further includes:
acquiring a preheating state corresponding to any hot spot url;
calculating to obtain a resource preheating success rate based on the preheating state corresponding to any one of the hot spot url;
and alarming the hot spot resources with the resource preheating success rate smaller than a preset success rate threshold.
The invention also provides a resource preheating device of the CDN, which comprises:
The acquisition module is used for acquiring the resource access log;
the determining module is used for carrying out resource access accumulation on the resource access log and determining each hotspot url according to the accumulation result;
The state analysis module is used for analyzing the states of all edge nodes in the CDN to obtain node state list information;
And the preheating module is used for distributing any hot point url to a corresponding edge node based on the node state list information so as to preheat the content of the hot point url in the edge node.
The invention also provides a computer device, which comprises a memory, a processor and computer readable instructions stored in the memory and capable of running on the processor, wherein the processor realizes the resource preheating method of the CDN when executing the computer readable instructions.
The present invention also provides one or more readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform a method of resource warming for a CDN network as described above.
The method, the device, the equipment and the storage medium for preheating the resources of the CDN comprise the following steps: acquiring a resource access log; accumulating the resource access log, and determining each hotspot url according to the accumulated result; analyzing the states of all edge nodes in the CDN to obtain node state list information; and distributing any hot url to a corresponding edge node based on the node state list information so as to preheat the content of the hot url in the edge node. According to the method and the system for predicting the hot spot resources in advance, the hot spot resources are predicted in advance according to indexes such as access duty ratio improvement of the resources, so that the triggering of the preheating task can be completed based on the quantity statistics of the CDN network, automatic preferential selection can be completed for the hot spot resource issuing node, the influence of excessive node resource load on the service quality of the existing network is avoided, the service quality of clients can be improved rapidly, and the preheating resources are comprehensive.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart illustrating a method for resource pre-heating in CDN according to an embodiment of the invention;
FIG. 2 is a schematic diagram illustrating a configuration of a CDN resource pre-heating device according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a computer device in accordance with an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terminology used in the one or more embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the invention. As used in one or more embodiments of the invention, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present invention refers to and encompasses any or all possible combinations of one or more of the associated listed items.
In an embodiment, as shown in fig. 1, fig. 1 is a flow chart of a method for preheating resources of a CDN network according to an embodiment of the present invention, and the embodiment of the present invention provides a method for preheating resources of a CDN network, including the following steps:
Step S11, obtaining a resource access log;
The resource access log refers to a log file in which client request information received by the edge node is recorded. In a network service, the edge node records information about each client request, such as the requested resource path, the time of the request, the client IP address and HTTP status code, etc. This information is recorded in the access log for subsequent analysis and monitoring.
Step S12, carrying out resource access accumulation on the resource access log, and determining each hotspot url according to an accumulation result;
Note that url generally refers to Uniform Resource Locator used when a network request is initiated. url is a string used to identify and locate resources on the internet. When a web site is entered in a browser or a network request is initiated by an application, it is actually the resource on the particular url being requested.
Specifically, based on the resource access log, the cumulative results such as the total number of requests of url resources or the total flow of requests in a preset period are counted, and further the url with the total number of requests of url being greater than a preset number threshold or the total flow of requests being greater than a preset flow threshold is used as the hotspot url, so that the advanced prediction of the hotspot resources is realized.
Step S13, analyzing the states of all edge nodes in the CDN to obtain node state list information;
The resource load information generally relates to indicators such as CPU utilization, memory usage, network traffic, and the like. By synchronizing the resource load information in real time, the management center can monitor the running state of the system, discover the resource bottleneck and abnormal conditions in time and take corresponding adjustment measures.
Specifically, in the present embodiment, it is required to monitor resource load information of the edge node, for example, monitor service process state information including service component operation state information. Monitoring node hardware resource load state information, including resource load state synchronization of a disk, IP, memory and the like; and monitoring node service traffic load information including statistical service traffic data information and the like. And carrying out health degree analysis on each edge node by combining the resource load information to obtain a scoring result corresponding to each edge node, and further comparing a preset scoring threshold value with the scoring result of any edge node so as to select the edge node with the scoring result larger than the preset scoring threshold value, and further forming the node state list information based on the selected edge node and the scoring result thereof.
Step S14, distributing any of the hot spot urls to the corresponding edge nodes based on the node status list information, so as to perform content preheating on the hot spot url in the edge nodes.
Specifically, based on domain name information of the hot spot url, selecting each target node associated with configuration from each edge node, and further, based on the node state list information, determining a scoring result corresponding to each target node, thereby selecting an edge node with the highest scoring result as the edge node corresponding to the hot spot url, distributing the hot spot url to the corresponding edge node, and further, carrying out content preheating on the hot spot url in the edge node. The more detailed preheating steps are as follows: the Agent receives the issued batch request and synchronously responds to the receiving result; analyzing by using an asynchronous thread to obtain the hot point url; a multithreaded simulation client initiates a request for accessing the hotspot url; the nginx receives the url request and forwards the url request to the ats (Apache Traffic Server) proxy server, and the ats proxy server initiates a resource request to the source station and caches the content of the response. And carrying out retry for a certain number of times when the preheating fails, thereby completing the preheating of url content.
The embodiment of the invention comprises the following steps: acquiring a resource access log; accumulating the resource access log, and determining each hotspot url according to the accumulated result; analyzing the states of all edge nodes in the CDN to obtain node state list information; and distributing any hot url to a corresponding edge node based on the node state list information so as to preheat the content of the hot url in the edge node. According to the method and the system for predicting the hot spot resources in advance, the hot spot resources are predicted in advance according to indexes such as access duty ratio improvement of the resources, so that the triggering of the preheating task can be completed based on the quantity statistics of the CDN network, automatic preferential selection can be completed for the hot spot resource issuing node, the influence of excessive node resource load on the service quality of the existing network is avoided, the service quality of clients can be improved rapidly, and the preheating resources are comprehensive.
In one embodiment of the present invention, the performing resource access accumulation on the resource access log, and determining each hotspot url according to the accumulation result includes:
based on the resource access log, periodically accumulating the total number of requests or the total flow of requests of each url; and if the total number of the requests of the url is larger than a preset number threshold or the total flow of the requests is larger than a preset flow threshold, taking the url as the hot spot url.
Specifically, in the resource access log, accumulating the total number of requests or the total flow of the requests of each url in a preset period, and comparing the total number of requests of the urls with a preset number threshold, or comparing the total flow of the requests with a preset flow threshold, and if the total number of requests of the urls is greater than the preset number threshold or the total flow of the requests is greater than the preset flow threshold, taking the urls as the hot point urls. For example: the rules for request quantity setting are as follows: when the number of requests is greater than a preset number threshold for a fixed period of time within a range of requested resources, the configurable parameters include, but are not limited to: range (node, province), period (1 minute, 5 minutes), preset number (percentage). The rules for requesting traffic settings are as follows: and when the request resource is in a certain range of fixed period, the request flow is larger than a preset flow threshold. The configuration parameters are as follows: range (node, province), period (1 minute, 5 minutes), preset flow threshold (flow).
According to the embodiment of the invention, the hot spot resources are predicted in advance according to the indexes such as the access duty ratio improvement of the resources, and the like, so that the triggering of the preheating task is actively completed through the real-time analysis of the hot spot resources.
In one embodiment of the present invention, the analyzing the state of each edge node in the CDN network to obtain node state list information includes:
Acquiring service process state information, node hardware resource load state information and node service traffic load state information of any edge node in a preset period; scoring the health degree of any edge node based on service process state information, node hardware resource load state information and node service traffic load state information corresponding to any edge node to obtain a scoring result of any edge node; and selecting and obtaining the node state list information based on a preset score threshold and a scoring result of any edge node.
Specifically, in order to avoid excessive consumption or unbalanced use of resources to improve the overall resource utilization efficiency, in this embodiment, monitoring the running state information of each service component of the edge node in a preset period, that is, whether each service process in the system runs normally, includes checking the health status of the service component. The load state information of the hardware resources of the node can also be monitored, namely, the use condition of the hardware resources of the node can be checked in real time, wherein the hardware resources of the monitoring node comprise a disk, an IP (possibly referred to as a network interface), a memory and the like. In addition, node service traffic load status information can be monitored, i.e. the system can record the service traffic of the node in detail so as to know the load condition of the node in different time periods. Further, the health degree of the edge node is scored by combining the service process state information, the node hardware resource load state information and the node service traffic load state information corresponding to the edge node, so that a scoring result of any edge node is obtained, wherein the score is possibly calculated according to a set rule and a weight, and the overall health condition of the node is reflected. And comparing the score threshold value with the score result of any one of the edge nodes to select the edge node with the score result larger than the score threshold value, thereby forming the node state list information. It is ensured that only well-conditioned nodes are selected to perform subsequent tasks or to provide services.
According to the embodiment of the invention, the state of the edge node is analyzed, so that the state health of the edge node can be preferentially selected in combination in the process of issuing the hot spot resource to the edge node, and the influence of the excessive load of the node resource on the service quality of the existing network is avoided.
In one embodiment of the present invention, the distributing any one of the hot spot url to a corresponding edge node based on the node status list information includes:
for any of the hot spots url: selecting each target node with configuration association from each edge node based on the domain name information of the hotspot url; and ordering all the target nodes based on the node state list information to obtain edge nodes corresponding to the hot point url, so as to distribute the hot point url to the corresponding edge nodes.
The distributing the hotspot url to the corresponding edge node includes: grouping and aggregating all hot point url belonging to the same edge node; and sending the aggregated hot point url to the corresponding edge node in batches through multiple threads.
Specifically, according to the domain name information of the hotspot url, determining which edge nodes are required to process the request related to the domain name, wherein the domain name information refers to the domain name to which url belongs. And sorting according to the grading result of each target node based on the node state list information, so as to select the target node with the highest grading result as the edge node corresponding to the hot spot url. Further, grouping and aggregating all the hot spot urls belonging to the same edge node, and further issuing all the aggregated hot spot urls to the corresponding edge node in batches in a multithreading mode, so that the sending efficiency of the request can be improved, and meanwhile, the request can reach the edge node in time.
Furthermore, if some requests fail to be sent, the system may perform a certain number of retries. This means that the system has an error handling mechanism that can cope with network anomalies or target node unavailability, etc., ensuring that the request can eventually be successfully sent to the edge node. After all requests are successfully sent to the corresponding edge node, the system completes the url sending task.
According to the embodiment of the invention, the edge node for processing the request related to the domain name is determined according to the domain name information of url, so that the preferred selection is performed by combining the state health degree of the edge node, and the influence of the excessive load of node resources on the service quality of the existing network is avoided.
In one embodiment of the present invention, after the content preheating of the hotspot url in the edge node, the method further includes:
Acquiring a preheating state corresponding to any hot spot url; calculating to obtain a resource preheating success rate based on the preheating state corresponding to any one of the hot spot url; and alarming the hot spot resources with the resource preheating success rate smaller than a preset success rate threshold.
Specifically, reporting an asynchronous initiation state after url preheating is completed, acquiring a preheating state corresponding to any hotspot url, and in addition, retrying for a certain number of times after reporting fails; and further, calculating a resource preheating success rate based on the preheating state corresponding to any one of the hot spot url, for example, calculating a resource preheating success rate according to the number of successful nodes and the number of failed nodes, and further, alarming the hot spot resource with the resource preheating success rate smaller than a preset success rate threshold.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In an embodiment, a device for preheating resources of a CDN is provided, where the device for preheating resources of the CDN corresponds to the method for preheating resources of the CDN in one-to-one manner in the foregoing embodiment. Fig. 2 is a schematic structural diagram of a CDN network resource preheating device according to an embodiment of the present invention, where the CDN network resource preheating device includes:
an acquisition module 21, configured to acquire a resource access log;
a determining module 22, configured to perform resource access accumulation on the resource access log, and determine each hotspot url according to the accumulation result;
The state analysis module 23 is configured to analyze the states of the edge nodes in the CDN network to obtain node state list information;
And the preheating module 24 is configured to distribute any one of the hot spot url to a corresponding edge node based on the node status list information, so as to preheat the content of the hot spot url in the edge node.
The determining module 22 is further configured to:
Based on the resource access log, periodically accumulating the total number of requests or the total flow of requests of each url;
And if the total number of the requests of the url is larger than a preset number threshold or the total flow of the requests is larger than a preset flow threshold, taking the url as the hot spot url.
The state analysis module 23 is further configured to:
Acquiring service process state information, node hardware resource load state information and node service traffic load state information of any edge node in a preset period;
Scoring the health degree of any edge node based on service process state information, node hardware resource load state information and node service traffic load state information corresponding to any edge node to obtain a scoring result of any edge node;
and selecting and obtaining the node state list information based on a preset score threshold and a scoring result of any edge node.
The preheating module 24 is also configured to:
for any of the hot spots url:
selecting each target node with configuration association from each edge node based on the domain name information of the hotspot url;
and ordering all the target nodes based on the node state list information to obtain edge nodes corresponding to the hot point url, so as to distribute the hot point url to the corresponding edge nodes.
The preheating module 24 is also configured to:
grouping and aggregating all hot point url belonging to the same edge node;
and sending the aggregated hot point url to the corresponding edge node in batches through multiple threads.
The preheating module 24 is also configured to:
analyzing by using an asynchronous thread to obtain the hot point url;
a multithreaded simulation client initiates a request for accessing the hotspot url;
and when the nginx receives the request, forwarding the request to a proxy server, and initiating a resource request to a source station by the proxy server and caching the response content to finish the content preheating of the hot url.
The resource preheating device of the CDN further comprises:
acquiring a preheating state corresponding to any hot spot url;
calculating to obtain a resource preheating success rate based on the preheating state corresponding to any one of the hot spot url;
and alarming the hot spot resources with the resource preheating success rate smaller than a preset success rate threshold.
The specific limitation of the resource preheating device of the CDN network may be referred to the limitation of the resource preheating method of the CDN network hereinabove, and will not be described herein. All or part of the modules in the resource preheating device of the CDN can be realized by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, and an internal structure thereof may be shown in fig. 3, and fig. 3 is a schematic diagram of the computer device in an embodiment of the present invention. The computer device includes a processor, a memory, a network interface, and a database connected by a device bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium, an internal memory. The readable storage medium stores operating means, computer readable instructions, and a database. The internal memory provides an environment for the execution of operating devices and computer-readable instructions in a readable storage medium. The database of the computer equipment is used for storing data related to the resource preheating method of the CDN. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer readable instructions, when executed by a processor, implement a method for resource warming of a CDN network. The readable storage medium provided by the present embodiment includes a nonvolatile readable storage medium and a volatile readable storage medium.
In one embodiment, a computer device is provided, which may be a terminal device, and the internal structure thereof may be as shown in fig. 3. The computer device includes a processor, a memory, and a network interface connected by a device bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer readable instructions. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer readable instructions, when executed by a processor, implement a method for resource warming of a CDN network. The readable storage medium provided by the present embodiment includes a nonvolatile readable storage medium and a volatile readable storage medium.
In one embodiment, a computer device is provided that includes a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, the processor implementing the steps of a method for preheating resources of a CDN network as described above when executing the computer readable instructions. The method comprises the following steps: acquiring a resource access log; accumulating the resource access log, and determining each hotspot url according to the accumulated result; analyzing the states of all edge nodes in the CDN to obtain node state list information; and distributing any hot url to a corresponding edge node based on the node state list information so as to preheat the content of the hot url in the edge node.
In an embodiment, a readable storage medium is provided, the readable storage medium storing computer readable instructions which, when executed by a processor, implement the steps of a method for preheating resources of a CDN network as described above. Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by instructing the associated hardware by computer readable instructions stored in a non-volatile readable storage medium or a volatile readable storage medium, which when executed may comprise the above described embodiment methods. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (6)
1. The resource preheating method for the CDN is characterized by comprising the following steps of:
Acquiring a resource access log;
Accumulating the resource access log, and determining each hotspot url according to the accumulated result;
Analyzing the states of all edge nodes in the CDN to obtain node state list information;
Distributing any hot url to a corresponding edge node based on the node state list information so as to preheat the content of the hot url in the edge node;
Analyzing the states of all edge nodes in the CDN to obtain node state list information, wherein the node state list information comprises the following steps:
Acquiring service process state information, node hardware resource load state information and node service traffic load state information of any edge node in a preset period;
Scoring the health degree of any edge node based on service process state information, node hardware resource load state information and node service traffic load state information corresponding to any edge node to obtain a scoring result of any edge node;
Selecting and obtaining the node state list information based on a preset score threshold value and a scoring result of any edge node;
The distributing any one of the hot spot url to a corresponding edge node based on the node status list information includes:
for any of the hot spots url:
selecting each target node with configuration association from each edge node based on the domain name information of the hotspot url;
Sorting all the target nodes based on the node state list information to obtain edge nodes corresponding to the hot point url, so as to distribute the hot point url to the corresponding edge nodes;
the distributing the hotspot url to the corresponding edge node includes:
grouping and aggregating all hot point url belonging to the same edge node;
Sending the aggregated hot point url batch to a corresponding edge node through multiple threads;
the content preheating of the hot url in the edge node includes:
analyzing by using an asynchronous thread to obtain the hot point url;
a multithreaded simulation client initiates a request for accessing the hotspot url;
and when the nginx receives the request, forwarding the request to a proxy server, and initiating a resource request to a source station by the proxy server and caching the response content to finish the content preheating of the hot url.
2. The method for preheating resources of a CDN network of claim 1 wherein the accumulated result includes a total number of requests or a total flow of requests for url;
the step of accumulating the resource access log, and determining each hotspot url according to the accumulated result, includes:
Based on the resource access log, periodically accumulating the total number of requests or the total flow of requests of each url;
And if the total number of the requests of the url is larger than a preset number threshold or the total flow of the requests is larger than a preset flow threshold, taking the url as the hot spot url.
3. The method for preheating resources of a CDN network of claim 1, further comprising, after the content preheating of the hotspot url in the edge node:
acquiring a preheating state corresponding to any hot spot url;
calculating to obtain a resource preheating success rate based on the preheating state corresponding to any one of the hot spot url;
and alarming the hot spot resources with the resource preheating success rate smaller than a preset success rate threshold.
4. A resource preheating device for a CDN network, comprising:
The acquisition module is used for acquiring the resource access log;
the determining module is used for carrying out resource access accumulation on the resource access log and determining each hotspot url according to the accumulation result;
The state analysis module is used for analyzing the states of all edge nodes in the CDN to obtain node state list information;
The preheating module is used for distributing any hot point url to a corresponding edge node based on the node state list information so as to preheat the content of the hot point url in the edge node;
Analyzing the states of all edge nodes in the CDN to obtain node state list information, wherein the node state list information comprises the following steps:
Acquiring service process state information, node hardware resource load state information and node service traffic load state information of any edge node in a preset period;
Scoring the health degree of any edge node based on service process state information, node hardware resource load state information and node service traffic load state information corresponding to any edge node to obtain a scoring result of any edge node;
Selecting and obtaining the node state list information based on a preset score threshold value and a scoring result of any edge node;
The distributing any one of the hot spot url to a corresponding edge node based on the node status list information includes:
for any of the hot spots url:
selecting each target node with configuration association from each edge node based on the domain name information of the hotspot url;
Sorting all the target nodes based on the node state list information to obtain edge nodes corresponding to the hot point url, so as to distribute the hot point url to the corresponding edge nodes;
the distributing the hotspot url to the corresponding edge node includes:
grouping and aggregating all hot point url belonging to the same edge node;
Sending the aggregated hot point url batch to a corresponding edge node through multiple threads;
the content preheating of the hot url in the edge node includes:
analyzing by using an asynchronous thread to obtain the hot point url;
a multithreaded simulation client initiates a request for accessing the hotspot url;
and when the nginx receives the request, forwarding the request to a proxy server, and initiating a resource request to a source station by the proxy server and caching the response content to finish the content preheating of the hot url.
5. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the method of resource warming of the CDN network of any one of claims 1 to 3 when executing the program.
6. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a method of resource warming of a CDN network as claimed in any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410239111.1A CN117834730B (en) | 2024-03-04 | 2024-03-04 | CDN (content delivery network) resource preheating method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410239111.1A CN117834730B (en) | 2024-03-04 | 2024-03-04 | CDN (content delivery network) resource preheating method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117834730A CN117834730A (en) | 2024-04-05 |
| CN117834730B true CN117834730B (en) | 2024-05-14 |
Family
ID=90511827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410239111.1A Active CN117834730B (en) | 2024-03-04 | 2024-03-04 | CDN (content delivery network) resource preheating method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117834730B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107463342A (en) * | 2017-08-28 | 2017-12-12 | 北京奇艺世纪科技有限公司 | A kind of storage method and device of CDN fringe nodes file |
| CN111614736A (en) * | 2020-04-30 | 2020-09-01 | 北京金山云网络技术有限公司 | Network content resource scheduling method, domain name scheduling server and electronic equipment |
| CN115002132A (en) * | 2022-05-23 | 2022-09-02 | 苏州思萃工业互联网技术研究所有限公司 | Distribution method, system and computer equipment for PCDN network pre-caching resource |
| CN116016546A (en) * | 2022-11-28 | 2023-04-25 | 天翼云科技有限公司 | Method and system for preheating resource files in batches in CDN |
| CN116915841A (en) * | 2022-11-02 | 2023-10-20 | ***通信集团有限公司 | CDN edge node hot spot spreading method, device, equipment and storage medium |
-
2024
- 2024-03-04 CN CN202410239111.1A patent/CN117834730B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107463342A (en) * | 2017-08-28 | 2017-12-12 | 北京奇艺世纪科技有限公司 | A kind of storage method and device of CDN fringe nodes file |
| CN111614736A (en) * | 2020-04-30 | 2020-09-01 | 北京金山云网络技术有限公司 | Network content resource scheduling method, domain name scheduling server and electronic equipment |
| CN115002132A (en) * | 2022-05-23 | 2022-09-02 | 苏州思萃工业互联网技术研究所有限公司 | Distribution method, system and computer equipment for PCDN network pre-caching resource |
| CN116915841A (en) * | 2022-11-02 | 2023-10-20 | ***通信集团有限公司 | CDN edge node hot spot spreading method, device, equipment and storage medium |
| CN116016546A (en) * | 2022-11-28 | 2023-04-25 | 天翼云科技有限公司 | Method and system for preheating resource files in batches in CDN |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117834730A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112162865B (en) | Scheduling method and device of server and server | |
| CN107426302B (en) | Access scheduling method, device, system, terminal, server and storage medium | |
| CN109412870B (en) | Alarm monitoring method and platform, server and storage medium | |
| US6397359B1 (en) | Methods, systems and computer program products for scheduled network performance testing | |
| US7613768B2 (en) | Method and system for managing a streaming media service | |
| CN109246229A (en) | A kind of method and apparatus of distribution resource acquisition request | |
| US8200842B1 (en) | Automatic traffic control using dynamic DNS update | |
| CN106254561B (en) | real-time off-line downloading method and system for network resource file | |
| CN109981805A (en) | A kind of method and device of domain name mapping | |
| US11025709B2 (en) | Load processing method and apparatus | |
| US11637902B2 (en) | Curating proxy server pools | |
| CN105959358A (en) | CDN server and method of CDN server of caching data | |
| CN108446172A (en) | Data transfer method, apparatus, computer equipment and storage medium | |
| CN108200132A (en) | Resource acquiring method, device, equipment and computer readable storage medium | |
| US9600251B1 (en) | Enhancing API service schemes | |
| CN111917898A (en) | Resource scheduling policy adjusting method and device | |
| CN117834730B (en) | CDN (content delivery network) resource preheating method, device, equipment and storage medium | |
| CN107908730B (en) | Method and device for downloading data | |
| US11606415B2 (en) | Method, apparatus and system for processing an access request in a content delivery system | |
| CN115208955A (en) | Resource request processing method and device, computer equipment and medium | |
| CN107846294B (en) | Main/standby service switching method and device | |
| CN105893150B (en) | Interface calling frequency control method and device and interface calling request processing method and device | |
| US10031799B1 (en) | Auditor for automated tuning of impairment remediation | |
| CN110891077A (en) | CDN node detection method and device | |
| CN112131198B (en) | Log analysis method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |