CN110691127B - Data acquisition control system, data acquisition control method and related device - Google Patents

Data acquisition control system, data acquisition control method and related device Download PDF

Info

Publication number
CN110691127B
CN110691127B CN201910911297.XA CN201910911297A CN110691127B CN 110691127 B CN110691127 B CN 110691127B CN 201910911297 A CN201910911297 A CN 201910911297A CN 110691127 B CN110691127 B CN 110691127B
Authority
CN
China
Prior art keywords
data
task
edge computing
request instruction
data acquisition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910911297.XA
Other languages
Chinese (zh)
Other versions
CN110691127A (en
Inventor
田江明
李欣
于亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN201910911297.XA priority Critical patent/CN110691127B/en
Publication of CN110691127A publication Critical patent/CN110691127A/en
Application granted granted Critical
Publication of CN110691127B publication Critical patent/CN110691127B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1008Server selection for load balancing based on parameters of servers, e.g. available memory or workload
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • Computer And Data Communications (AREA)

Abstract

The invention provides a data acquisition control system, a data acquisition control method and a related device, wherein an edge computing device sends a task data acquisition request instruction to a task server, the task server responds to the task data acquisition request instruction and sends task data generated by the task server to the edge computing device, the edge computing device generates a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data, the edge computing device sends a data acquisition request instruction to a data source server, the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area, the edge computing device receives and stores the data to be distributed sent by the data source server so as to undertake part of service requests of the data source server through the edge computing device, so that the peak value of access flow of the data source server is reduced, when the peak value of the access traffic is reduced, the bandwidth cost calculated according to the peak value of the access traffic is also reduced.

Description

Data acquisition control system, data acquisition control method and related device
Technical Field
The present invention relates to the field of data acquisition, and in particular, to a data acquisition control system, a data control method, and a related apparatus.
Background
With the continuous development of internet technology, more and more services can be provided by a data source server, different terminals can send service requests to the data source server so that the data source server provides services matched with the service requests, but the service provided by the data source server is influenced by the fact that the data source server receives too many service requests at the same time.
Disclosure of Invention
In view of the above problems, the present invention is proposed to provide a data acquisition control system, a data control method and a related apparatus that overcome or at least partially solve the above problems, and the technical solutions are as follows:
the present invention provides a data acquisition control system, the system comprising: an edge computing device and a task server;
the edge computing device is used for sending a task data acquisition request instruction to the task server;
the task server is used for responding to the task data acquisition request instruction and sending the task data generated by the task server to the edge computing equipment;
the edge computing device is further configured to generate a data obtaining request instruction based on a download task used for obtaining data to be distributed in the task data, send the data obtaining request instruction to a data source server, where the data obtaining request instruction is used to trigger the data source server to obtain the data to be distributed matched with the data obtaining request instruction from a storage area, and receive and store the data to be distributed sent by the data source server.
The present invention also provides an edge computing device, comprising: a processor, a memory, and a communication interface;
the processor is used for acquiring task data sent by a task server, generating a data acquisition request instruction based on a download task for acquiring data to be distributed in the task data, and sending the data acquisition request instruction to a data source server through the communication interface, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area;
the processor is further configured to receive the data to be distributed sent by the data source server through the communication interface, and store the data to be distributed in the memory.
The present invention also provides a task server, including: a processor and a communication interface;
the processor is used for generating task data and receiving a task data acquisition request instruction sent by the edge computing equipment through the communication interface;
the processor is further configured to send the task data to the edge computing device through the communication interface in response to the task data obtaining request instruction, where the task data is used to instruct the edge computing device to obtain data to be distributed from the data source server.
The invention also provides a data acquisition control method, which comprises the following steps:
the edge computing equipment sends a task data acquisition request instruction to the task server;
the task server responds to the task data acquisition request instruction and sends the task data generated by the task server to the edge computing equipment;
the edge computing equipment generates a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data;
the edge computing device sends the data acquisition request instruction to a data source server, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area;
and the edge computing equipment receives and stores the data to be distributed sent by the data source server.
The invention provides a data acquisition control method, which is applied to edge computing equipment and comprises the following steps:
acquiring task data sent by a task server;
generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data;
sending the data acquisition request instruction to a data source server, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area;
and receiving and storing the data to be distributed sent by the data source server.
Preferably, the sending the data obtaining request instruction to the data source server includes:
and sequentially sending data acquisition request instructions to the data source server based on preset interval time.
Preferably, in response to sending the data obtaining request instruction to the data source server, the method further includes:
recording data downloading time, wherein the data downloading time is the time consumed from the time of sending the data acquisition request instruction to the time of finishing storing the data to be distributed;
if the data downloading time is less than or equal to a preset interval time, postponing a first waiting time after finishing storing the data to be distributed to execute the step of sending the data acquisition request instruction to a data source server, wherein the first waiting time is the difference between the preset interval time and the data downloading time;
and if the data downloading time is greater than the preset interval time, delaying a preset second waiting time after the moment of storing the data to be distributed is finished, and executing the step of sending the data acquisition request instruction to the data source server.
Preferably, the acquiring task data sent by the task server includes:
if the current moment of the edge computing equipment is within a preset data acquisition time period of the edge computing equipment, sending a task data acquisition request instruction to the task server;
and receiving the task data sent by the task server.
Preferably, after receiving the task data sent by the task server, the method further includes:
when the edge computing device runs for a first interval time after the moment of receiving the task data, judging whether the current moment corresponding to the first interval time after the moment of receiving the task data runs is within a preset data acquisition time period of the edge computing device;
if the current moment is within a preset data acquisition time period of the edge computing device, executing a step of sending a task data acquisition request instruction to the task server;
and if the current time is not within the preset data table acquisition time period of the edge computing equipment, forbidding sending a task data acquisition request instruction to the task server at the current time.
Preferably, after receiving the task data sent by the task server, the method further includes:
if the task data is empty, when the edge computing device operates a second interval duration after the time when the received task data is empty, judging whether the current time corresponding to the second interval duration after the time when the received task data is empty is within a preset data acquisition time period of the edge computing device;
if the task data are within the preset data acquisition time period of the edge computing equipment, a step of sending a task data acquisition request instruction to the task server is executed;
and if the current time is not within the preset data acquisition time period of the edge computing device, postponing the current time backwards for a first interval duration to judge whether the time corresponding to the first interval duration postponed backwards from the current time is within the preset data acquisition time period of the edge computing device.
Preferably, the acquiring task data sent by the task server includes: and if the total data volume of the data to be distributed stored in the edge computing equipment is less than the preset data volume, acquiring the task data sent by the task server.
Preferably, the method further comprises: and if the total data volume of the data to be distributed stored in the edge computing equipment is larger than or equal to the preset data volume, forbidding to acquire the task data sent by the task server.
The invention provides a data acquisition control method, which is applied to a task server and comprises the following steps:
generating task data;
receiving a task data acquisition request instruction sent by edge computing equipment;
and responding to the task data acquisition request instruction, and sending the task data to the edge computing equipment, wherein the task data is used for instructing the edge computing equipment to acquire the data to be distributed from the data source server.
Preferably, the sending the task data to the edge computing device in response to the task data obtaining request instruction includes:
responding to the task data acquisition request instruction, and if the moment of receiving the task data acquisition request instruction is within a preset task data issuing time period, sending the task data to the edge computing equipment.
Preferably, the sending the task data to the edge computing device in response to the task data obtaining request instruction includes:
extracting identification information of the edge computing device, the data volume to be stored of the edge computing device and the sending time of the task data acquisition request instruction from the task data acquisition request instruction;
acquiring preset task data within a preset task data issuing time period corresponding to the sending time;
and sending task data matched with the data volume to be stored to the edge computing equipment based on the identification information of the edge computing equipment and the data volume to be stored of the edge computing equipment.
The present invention also provides a data acquisition control system, the system comprising: an edge computing device and a task server;
the edge computing device is used for sending a task data acquisition request instruction to the task server;
the task server is used for responding to the task data acquisition request instruction and sending the task data generated by the task server to the edge computing equipment;
the edge computing device is configured to generate a data obtaining request instruction based on a download task for obtaining data to be distributed in the task data, send the data obtaining request instruction to a data source server, where the data obtaining request instruction is used to trigger the data source server to obtain the data to be distributed matching with the data obtaining request instruction from a storage area, and receive and store the data to be distributed sent by the data source server.
The present invention also provides a data acquisition control apparatus, which is applied to an edge computing device, and includes:
the acquisition unit is used for acquiring task data sent by the task server;
the generating unit is used for generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data;
a sending unit, configured to send the data obtaining request instruction to a data source server, where the data obtaining request instruction is used to trigger the data source server to obtain, from a storage area, the data to be distributed that is matched with the data obtaining request instruction;
and the receiving and storing unit is used for receiving and storing the data to be distributed sent by the data source server.
The invention also provides a data acquisition control device, which is applied to the task server and comprises the following components:
a generation unit configured to generate task data;
the receiving unit is used for receiving a task data acquisition request instruction sent by the edge computing equipment;
and the response unit is used for responding to the task data acquisition request instruction and sending the task data to the edge computing equipment, wherein the task data is used for indicating the edge computing equipment to acquire the data to be distributed from the data source server.
By the technical scheme, the edge computing device sends a task data acquisition request instruction to the task server, the task server responds to the task data acquisition request instruction and sends task data generated by the task server to the edge computing device, the edge computing device generates a data acquisition request instruction based on a download task for acquiring data to be distributed in the task data, the edge computing device sends the data acquisition request instruction to the data source server, the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from the storage area, the edge computing device receives and stores the data to be distributed sent by the data source server, so that the data to be distributed in the storage area corresponding to the data source server can be stored by the edge computing device, and a service request for requesting the data to be distributed sent to the data source server can be responded by the edge computing device, the edge computing equipment bears part of the service requests of the data source server, so that the response speed of the service requests is improved. And the peak value of the access flow for triggering the data source server to provide the service is reduced by adopting a mode that the edge computing equipment bears part of the service request of the data source server, the bandwidth cost is calculated on the basis of the peak value of the access flow, and when the peak value of the access flow is reduced, the bandwidth cost calculated according to the peak value of the access flow is also reduced.
The edge computing device and the task server can control the transmission of the task data to control the transmission of a data acquisition request instruction to the data source server by the edge computing device and the task server, so as to realize the control of the distribution speed of the data to be distributed, the edge computing device can also control the transmission of two adjacent data acquisition request instructions, the edge computing device receives and stores the data to be distributed from the data source server at intervals of the transmission of the two adjacent data acquisition request instructions, thereby realizing the control of the download speed of the data to be distributed by controlling the transmission of the data acquisition request instructions, realizing the control of the total download data quantity of the edge computing device by controlling the distribution speed and the download speed of the data to be distributed, and leading the total download data quantity of the edge computing device in a download period to be close to the preset total quantity, the total amount of the downloaded data of different edge computing devices in the same downloading period is the same, the total amount of the downloaded data of different edge computing devices can be similar through the control, which means that the services which can be provided by different edge computing devices are similar, the different edge computing devices can provide the similar services when requesting, and the load balance among different edge computing devices is realized.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a diagram illustrating an architecture of a data acquisition system provided by an embodiment of the present invention;
fig. 2 is a signaling diagram illustrating a data acquisition control method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a preset interval time for processing two adjacent data acquisition request instructions by the edge computing device according to the embodiment of the present invention;
FIG. 4 is a flow chart illustrating another data acquisition control method provided by an embodiment of the invention;
fig. 5 is a flowchart illustrating a further data acquisition control method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram illustrating a data acquisition control apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another data acquisition control apparatus according to an embodiment of the present invention.
Detailed Description
To solve the problems in the prior art, the present embodiment introduces a data acquisition control system, whose architecture is shown in fig. 1, and may include: an edge computing device 100 and a task server 200. The edge computing device 100 may provide services as a data source server, but the edge computing device 100 needs To obtain data corresponding To the services from the data source server 300 capable of providing the services, for example, in order To provide a Point-To-Point (P2P) upload service, the edge computing device 100 needs To download data required for the P2P upload service from a Content Delivery Network (CDN) server (a data source server).
For example, a router may serve as the edge computing device 100, the data acquisition control method provided in this embodiment is implemented by the router, data in the data source server is stored in a storage space corresponding to the router in advance, when a user operates a mobile client or a PC client to acquire a service corresponding to the part of data, the router may directly provide the service based on the part of data stored in the storage space corresponding to the router, and of course, a device having a function similar to that of the router may serve as the edge computing device 100, except for the router.
The task server 200 is configured to control the edge computing devices 100 to obtain data corresponding to a service, and in this embodiment, the task server 200 may simultaneously control at least one edge computing device 100 to control the edge computing device 100 to obtain data from the data source server 300 by sending task data to the edge computing device 100.
Specifically, the edge computing device 100 is configured to send a task data obtaining request instruction to the task server; the task server 200 is configured to respond to the task data obtaining request instruction and send task data generated by the task server to the edge computing device; the edge computing device 100 is further configured to generate a data obtaining request instruction based on a download task for obtaining data to be distributed in the task data, send the data obtaining request instruction to the data source server, where the data obtaining request instruction is used to trigger the data source server to obtain data to be distributed matching with the data obtaining request instruction from the storage area, and receive and store the data to be distributed sent by the data source server.
The edge computing device and the task server of the present embodiment will be described in detail below with reference to the drawings. While the present embodiments are illustrated in the accompanying drawings, it should be understood that they may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiments to those skilled in the art.
As shown in fig. 2, which shows a signaling diagram of a data acquisition control method provided in an embodiment of the present invention, the signaling diagram may include the following steps:
s201: and the edge computing equipment sends a task data acquisition request instruction to the task server. Wherein the task data fetch request instruction indicates that the edge computing device can serve as a data source server and needs to be allocated by the task server to fetch which data (i.e., from which data source server to fetch data). In this embodiment, the edge computing device may send the task data obtaining request instruction after starting the to-be-distributed data obtaining function, for example, a rule in the following form is configured in the edge computing device:
<dl ms="on"
</dl>
and if the value of ms is off, the data acquisition function to be distributed is closed, and the task data acquisition request instruction is forbidden to be sent.
In addition, the embodiment may also control the time when the edge computing device sends the task data acquisition request instruction, and the edge computing device may set the time when the edge computing device is in an idle state as the time for controlling the task data acquisition request instruction, where the idle state may indicate that the edge computing device currently has enough idle resources or processing capacity to process the task data acquisition request, for example, the edge computing device does not have much interaction with the task server and also has enough network bandwidth to send the task data acquisition request.
The time for sending the task data acquisition request instruction may include at least one preset data acquisition time period or at least one data acquisition time point, as follows:
<dl<s t="aaaaaa-bbbbbb">
<s t="cccccc-dddddd">
<s t="eeeeee-ffffff">
</dl>
a plurality of preset data acquisition time periods are set in the edge computing device, and if the current time of the edge computing device is within any preset data acquisition time period of the plurality of time periods, a task data acquisition request instruction can be sent. Of course, the above-mentioned time point setting and the ms setting can be used in combination, and the embodiment will not be described.
S202: and the task server responds to the task data acquisition request instruction and sends the task data generated by the task server to the edge computing equipment. Wherein the task data is used to instruct the edge computing device to obtain the source of the data and obtain the data corresponding to which service (i.e. from which data source server to obtain the data).
The task data generated by the task server includes at least one download task, where the download task is used to obtain data to be distributed from the data source server, and the data to be distributed provides a device required by the service for the edge computing device, for example, the download task includes, but is not limited to, an identifier (such as an IP address) of the data source server and a service to be provided by the edge computing device, so as to obtain the data from the data source server through these parameters.
In this embodiment, the task server may respond to the task data obtaining request instruction when in an idle state, where the idle state may indicate that the task server currently has enough idle resources or processing capacity to process the task data obtaining request sent by the edge computing device, for example, the task server currently processes a small number of task data obtaining requests, and may also continue to process task data obtaining requests sent by other edge computing devices.
The task server processes the task data acquisition request explanation sent by the edge computing device when the task server has enough free resources or processing capacity: the method comprises the steps that the number of requests processed by a current task server is small, interaction with edge computing equipment is small, and at the moment, the bandwidth occupation of the task server is small, which means that the flow peak value of the task server is low, so that the edge computing equipment is controlled to obtain a task distribution list when the task server is in an idle state, the condition that the task server generates centralized processing requests and/or is requested to distribute high data volume is reduced, the bandwidth of the task server is prevented from being occupied in a centralized mode, the flow peak value of the task server can be reduced, and the condition that the bandwidth cost is too high due to the fact that the task server reaches the flow peak value is avoided. And if the situation that the task server generates centralized processing requests and/or the hit requests distribute high data volume can be reduced in this way, the processing of the task server cannot be hit within a certain time, and therefore the processing pressure of the task server can be reduced.
For example, if the configuration of the task server is higher, the task server has more requests currently processed, but it may also be considered that the task server with higher configuration has sufficient idle resources or processing capability, and whether the task server has sufficient idle resources or processing capability is not limited in this embodiment.
For the task server, a time for the task server to respond to the task data obtaining request instruction may be set, where the time may be at least one preset task data issuing time period and/or at least one time point, and if a time at which the task server receives the task data obtaining request instruction matches the set time, the task server sends the task data to the edge computing device in response to the task data obtaining request instruction, for example, the time at which the task server receives the task data obtaining request instruction is within the preset task data issuing time period, and sends the task data to the edge computing device. One way of presetting the task data issuing time period is as follows:
<PR P="xxx">
<TR BT="01:00:00"ET="01:29:00"Count="30"/>
<TR BT="01:30:00"ET="01:59:00"Count="55"/>
</PR>
PR represents a rule, different types of edge computing devices can be controlled differently by the rule as a device rule, P ═ xxx represents identification information of the edge computing devices, such as the types of the edge computing devices, so that a plurality of edge computing devices having the same type can share the same rule, TR is a time rule for setting a preset task data delivery time period for a task server to respond to a task data acquisition request instruction, BT is a start time of the preset task data delivery time period, and ET is an end time of the preset task data delivery time period. The rule may further set a maximum number of download tasks provided for the edge computing device within a preset task data delivery time period, where the Count is a single-time distribution amount, that is, the maximum number of download tasks provided for the edge computing device between the start time and the end time of a preset task data delivery time period.
S203: the edge computing device generates a data acquisition request instruction based on a download task for acquiring data to be distributed in the task data. The foregoing description refers to a download task for obtaining data to be distributed from a data source server, where the data to be distributed is a device required by the edge computing device to provide a service, for example, the download task includes, but is not limited to, an identifier (such as an IP address) of the data source server and a service to be provided by the edge computing device, so as to obtain the data from the data source server through these parameters.
Based on this, the data obtaining request instruction of this embodiment may carry the identifier of the data source server and the identifier of the corresponding service, so as to obtain the data matched with the service from the corresponding data source server.
S204: the edge computing device sends a data acquisition request instruction to the data source server, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area, and the storage area can be coupled and communicated with the data source server so that the data source server can acquire the data to be distributed from the storage area, for example, the storage area can be located in the data source server.
In this embodiment, the edge computing device may further set a preset rule for controlling the sending of the data obtaining request instruction, for example, the preset rule may define but is not limited to: the edge computing device sends the preset interval time of the data acquisition request instruction, and then the edge computing device needs to send the data acquisition request instruction to the data source server in sequence according to the requirement of the preset interval time when sending the data acquisition request instruction. The further edge computing device may send the data obtaining request commands in sequence at preset intervals in the preset data obtaining time period.
In this embodiment, in order to enable the data volumes of the data to be distributed received by different edge computing devices in the same time period to be close, the embodiment may set the same preset interval time and the same waiting time with a smaller value in all the edge computing devices in advance (depending on the actual requirement), if the time consumed by one edge computing device from sending the data acquisition request instruction to receiving the data to be distributed is less than or equal to the preset interval time, the time between the sending of two adjacent data acquisition request instructions by the edge computing device is the preset interval time, and if the time consumed by one edge computing device from sending the data acquisition request instruction to finishing storing the data to be distributed is greater than the preset interval time, postponing one waiting time to send the next data acquisition request instruction after the preset interval time, the time equivalent to the interval between the two adjacent data acquisition request instructions sent by the edge device is the sum of the preset interval time and the waiting time, and in order to shorten the interval between the two adjacent data acquisition request instructions, the waiting time needs to be reduced as much as possible, and another purpose of reducing the waiting time is to: the time spent by the edge computing device capable of sending the two adjacent data acquisition request instructions within the preset interval time and the time spent by the edge computing device incapable of sending the two adjacent data acquisition request instructions within the preset interval time for storing the same/similar data volume are shortened, so that the data volumes of the data to be distributed received by the edge computing devices with different downloading speeds within the same time period can be close to each other.
Taking the time period of one day as an example, for the edge computing devices with fast download, the time consumed by each data acquisition request instruction from sending to finishing storing the data to be distributed is less than or equal to the preset interval time, which indicates that the edge computing devices with fast download can finish processing the data acquisition request instruction within the preset interval time, so that the events of the edge computing devices processing each data acquisition request instruction are the preset interval time, and the data volumes downloaded by the edge computing devices with fast download within the same day can be close to each other or even the same; for the edge computing device with slow downloading, if the edge computing device with slow downloading can not complete the processing of the data obtaining request instruction within the preset interval time, it needs to wait for a relatively small waiting time to process the next data obtaining request instruction, so as to shorten the difference of the overall downloading speeds of the edge computing device with slow downloading and the edge computing device with fast downloading by setting a relatively small waiting time, and further reduce the difference of the data amounts downloaded by the edge computing device with slow downloading and the edge computing device with fast downloading within the preset time.
Based on the above analysis, one way to control the data acquisition request instruction transmission in this embodiment is as follows: the data downloading time is recorded in response to the data acquisition request instruction sent to the data source server, wherein the data downloading time is the time consumed from the data acquisition request instruction sent to the data source server to the end of storing the data to be distributed, and the data acquisition request instruction sent to the data source server in response means that the data downloading time is acquired by starting to immediately count at the moment of sending the data acquisition request instruction.
If the data downloading time is less than or equal to the preset interval time, delaying the first waiting time from the moment of finishing storing the data to be distributed to execute the step of sending the data acquisition request instruction to the data source server, wherein the first waiting time is the difference between the preset interval time and the data downloading time, so that the edge computing equipment which can process the data acquisition request instruction has the time length less than the preset interval time can send the next data acquisition request instruction only by waiting for the first waiting time after processing the data acquisition request instruction, and the time for sending the two adjacent data acquisition request instructions by the edge computing equipment which can download quickly is ensured to be the preset interval time.
If the data downloading time is greater than the preset interval time, delaying a preset second waiting time after the moment of storing the data to be distributed is finished to execute the step of sending the data acquisition request instruction to the data source server, so that for the edge computing device which cannot finish the data acquisition request instruction within the preset interval time, the next data acquisition request instruction needs to be sent by waiting the second waiting time, and the difference between the time of sending two adjacent data acquisition request instructions by the edge computing device which downloads quickly and the time of sending two adjacent data acquisition request instructions by the edge computing device which downloads slowly is smaller. The following description is made with reference to the accompanying drawings:
as shown in fig. 3, the preset interval time is denoted by td, the time from the sending of the data acquisition request instruction to the end of storing the data to be distributed by the edge computing device is denoted by time _ download, and the end of storing the data to be distributed explains that the edge computing device completes downloading the data from the storage area of the data source server, so that the time _ download may also denote the download time of the data to be distributed from the storage area of the data source server by the edge computing device, the time waiting from the end of storing the data to be distributed to the start of the next data acquisition request instruction is denoted by time _ wait, and if the time _ download is not greater than td, the time _ wait is the first wait time, and the time _ download and the time _ wait are added to obtain td. That is, if the edge computing device can complete one data acquisition request instruction in td, the interval time between the edge computing device processing two adjacent data acquisition request instructions needs to be td, during which the sum of time _ download and time _ wait is always td regardless of the change of time _ download, in which case the first wait time _ wait is td-time _ download, and the first wait time _ wait changes with the change of time _ download.
If the time _ download is greater than td, the time _ wait is a second waiting time, and the second waiting time is a preset time, so that the sum of the time _ download and the time _ wait is greater than td, therefore, in order to shorten the time interval for processing two adjacent data acquisition request instructions between edge computing devices with different download speeds, the preset second waiting time needs to be as small as possible, so as to reduce the waiting time for starting the next data acquisition request instruction by the edge computing device with the slower download speed, thereby accelerating the speed for processing two adjacent data acquisition request instructions by the edge computing device with the slower download speed, for example, the preset second waiting time is 2 seconds.
In the following, how to reduce the difference in data amount between different edge computing devices based on the data acquisition control method provided in this embodiment is described with reference to fig. 3, starting from an edge computing device with a fast download speed and an edge computing device with a slow download speed:
specifically, for the edge computing device with the fast download speed, the total time consumption of each data acquisition request instruction is td, so that the data volume downloaded by the edge computing device with the fast download speed in the preset time is basically the same; for the edge computing device with the slow downloading speed, after the data corresponding to a certain data obtaining request instruction is downloaded, a second waiting time with a shorter waiting time is needed, so that the overall waiting time of the edge computing device with the slow downloading speed in the preset time is reduced compared with that of the edge computing device with the fast downloading speed, and the interval time of the edge computing device with the slow downloading speed and the edge computing device with the fast downloading speed for processing two adjacent data obtaining request instructions is close to each other, and further the overall downloading speed difference of the edge computing device with the fast downloading speed and the edge computing device with the slow downloading speed in the preset time is reduced, so that the quantity difference between different edge computing devices is reduced, and the services which can be provided by different edge computing devices are close to each other due to the reduced data quantity difference.
In this embodiment, in order to reduce the data amount difference between different edge computing devices, the preset interval time td in different edge computing devices may be the same or close to each other, and the same preset second waiting time may also be the same or close to each other, for example, the second waiting time may be set by a technician according to the specific situations such as the pressure of distributing task data currently faced by the table server, the current remaining storage amount of different edge computing devices, and the download speed of different edge computing devices, so as to set a same or close second waiting time between different edge computing devices, and also for the interval time, the setting of the interval time and the preset second waiting time is not limited in this embodiment.
The preset interval time may be combined with the preset data acquisition time period as follows:
<dl s t="aaaaaa-bbbbbb",td=80>
<s t="cccccc-dddddd",td=70>
<s t="eeeeee-ffffff",td=75>
</dl>
and setting respective corresponding preset interval time in each preset data acquisition time period to control the processing process of the data acquisition request instruction based on the respective preset interval time in each preset data acquisition time period, which is not described in detail in this embodiment.
S205: and the edge computing equipment receives and stores the data to be distributed sent by the data source server.
By the technical scheme, the edge computing device sends a task data acquisition request instruction to the task server, the task server responds to the task data acquisition request instruction and sends task data generated by the task server to the edge computing device, the edge computing device generates a data acquisition request instruction based on a download task for acquiring data to be distributed in the task data, the edge computing device sends the data acquisition request instruction to the data source server, the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from the storage area, the edge computing device receives and stores the data to be distributed sent by the data source server, so that the data to be distributed in the storage area corresponding to the data source server can be stored by the edge computing device, and a service request for requesting the data to be distributed sent to the data source server can be responded by the edge computing device, the edge computing equipment bears part of the service requests of the data source server, so that the response speed of the service requests is improved. And the peak value of the access flow for triggering the data source server to provide the service is reduced by adopting a mode that the edge computing equipment bears part of the service request of the data source server, the bandwidth cost is calculated on the basis of the peak value of the access flow, and when the peak value of the access flow is reduced, the bandwidth cost calculated according to the peak value of the access flow is also reduced.
The edge computing device and the task server can control the transmission of the task data to control the transmission of a data acquisition request instruction to the data source server by the edge computing device and the task server, so as to realize the control of the distribution speed of the data to be distributed, the edge computing device can also control the transmission of two adjacent data acquisition request instructions, the edge computing device receives and stores the data to be distributed from the data source server at intervals of the transmission of the two adjacent data acquisition request instructions, thereby realizing the control of the download speed of the data to be distributed by controlling the transmission of the data acquisition request instructions, realizing the control of the total download data quantity of the edge computing device by controlling the distribution speed and the download speed of the data to be distributed, and leading the total download data quantity of the edge computing device in a download period to be close to the preset total quantity, the total amount of the downloaded data of different edge computing devices in the same downloading period is the same, the total amount of the downloaded data of different edge computing devices can be similar through the control, which means that the services which can be provided by different edge computing devices are similar, the different edge computing devices can provide the similar services when requesting, and the load balance among different edge computing devices is realized.
Referring to fig. 4, another data acquisition control method provided by the embodiment of the invention is shown, which is applied to an edge computing device, and includes the following steps:
s401: and acquiring task data sent by the task server. One way to obtain task data is to: the edge computing device sends a task data acquisition request instruction to the task server and receives task data sent by the task server, wherein the task data is obtained after the task server receives the task data acquisition request instruction and is used for indicating the source of the data to be distributed acquired by the edge computing device and acquiring data corresponding to services.
In this embodiment, the edge computing device may control sending of the task data obtaining request instruction, for example, a preset data obtaining time period is set in the edge computing device, and if the current time of the edge computing device is in the preset data obtaining time period, the task data obtaining request instruction is sent to the task server.
The preset data obtaining time period is used for controlling the task data obtaining request instruction sent each time, and this embodiment may also control the interval between two adjacent task data obtaining request instructions, and the control method includes, but is not limited to, the following control methods:
one way is as follows: when the edge computing device runs for a first interval time after the moment of receiving the task data, judging whether the current moment corresponding to the first interval time after the moment of receiving the task data runs is within a preset data acquisition time period of the edge computing device; if the current moment is within a preset data acquisition time period of the edge computing device, executing a step of sending a task data acquisition request instruction to a task server; and if the current moment is not within the preset data table acquisition time period of the edge computing device, forbidding sending a task data acquisition request instruction to the task server at the current moment.
That is, after receiving the task data each time, the first interval duration is operated, and then whether the current time is within the preset data acquisition time period is judged, if so, a task data acquisition request instruction can be sent to acquire the task data again; if not, the task data acquisition request instruction is prohibited from being sent, and at this time, the first interval duration can be operated again, and then whether the time is within the preset data acquisition time period or not can be judged.
For example, the first interval duration is r, the edge computing device determines whether the preset data acquisition time period is reached when sending the task data acquisition request instruction for the first time, sends the task data acquisition request instruction if the preset data acquisition time period is reached, and determines whether the preset data acquisition time period is reached again if the preset data acquisition time period is not reached. If the time for sending the task data acquisition request instruction for the first time is within the preset data acquisition time period, the r time length is operated to judge that a + r is not within the preset data acquisition time period (a is the time for sending the task data acquisition request instruction for the first time), the r time length is operated again to judge the a + r + r, and the like until the obtained time is within the preset data acquisition time period, the task data acquisition request instruction can be sent, the time for receiving the task data after the task data acquisition request instruction is sent at this time is recorded, and the time for sending the task data acquisition request instruction for the next time is determined by the time, the first interval time length and the preset data acquisition time period.
In another mode: if the task data is empty, when the edge computing device operates a second interval duration after the moment when the received task data is empty, judging whether the current moment corresponding to the second interval duration after the moment when the received task data is empty is within a preset data acquisition time period of the edge computing device; if the task data is in the preset data acquisition time period of the edge computing equipment, a step of sending a task data acquisition request instruction to the task server is executed; and if the current time is not within the preset data acquisition time period of the edge computing device, postponing the first interval time from the current time backwards to judge whether the time corresponding to the first interval time postponed from the current time backwards is within the preset data acquisition time period of the edge computing device.
Compared with the former way, in this embodiment, an interval for controlling two adjacent task data acquisition request instructions is sent from the task data to be empty or not to be empty, if the task data is empty, the second interval duration is postponed first, and then whether the task data is within the preset data acquisition time period is judged, if not, the first interval duration needs to be postponed; if the task data is not empty, control can be performed directly based on the last manner.
If the first interval duration is r and the second interval duration is g, if the task data received by the edge computing device for the 5 th time is empty, the moment when the edge computing device sends the task data acquisition request instruction for the 6 th time is the moment obtained by delaying g from a, namely a + g, if the time after delaying g is not within the preset data acquisition time period, then the time of sending the task data acquisition request instruction at the 6 th time needs to be delayed by r on the basis of a + g, namely, the time of sending the task data acquisition request command at the 6 th time is a + g + r, if the a + g + r is within the preset data acquisition time period, the a + g + r is determined as the time of sending the task data acquisition request command at the 6 th time, and sending a task data acquisition request instruction to the task server at the moment, and obtaining the 7 th moment of sending the task data acquisition request instruction based on the moment.
Based on the mode, the first interval duration, the second interval duration and at least one preset data acquisition time period are set, so that the time for the edge computing equipment to acquire the task data can be flexibly regulated and controlled according to different edge computing equipment, the pressure for the task server to distribute the task data in a centralized mode is reduced, the flow peak value of the task server side can be reduced, and the bandwidth cost is reduced.
The settings corresponding to the above-described modes are as follows:
<dl ms="on",g="5400",r="5400"
<s t="aaaaaa-bbbbbb">
<s t="cccccc-dddddd">
<s t="eeeeee-ffffff">
</dl>
ms represents whether the data acquisition function to be distributed is turned on, and please refer to the related method in the above embodiment, g is the second interval duration, r is the first interval duration, < s t ═ aaaa-bbbbbb > represents a preset data acquisition time period, t ═ aaaa-bbbb "records the corresponding time period, and takes the time [ aa: aa, bb: bb: bb ] from the beginning of the configuration at the time aa: aa (inclusive) to the end of the configuration at the time bb: bb (exclusive). If the value of ms is detected to be changed from off to on, the time point at which the value of ms is detected to be changed to on is taken as the time for sending the task data acquisition request instruction for the first time, and if the value of ms is detected to be on after the edge computing device is started, the starting time can also be taken as the time for sending the task data acquisition request instruction for the first time, the setting of r, g and t can be determined according to actual requirements, and the embodiment is not limited.
In this embodiment, the task data may also be obtained based on other manners, for example, a preset data amount is preset, where the preset data amount is used to indicate a maximum total data amount of the data to be distributed stored in the edge computing device, and if the total data amount of the data to be distributed stored in the edge computing device is smaller than the preset data amount, the task data sent by the task server is obtained; and if the total data volume of the data to be distributed stored in the edge computing equipment is larger than or equal to the preset data volume, forbidding to acquire the task data sent by the task server.
Of course, the first interval duration, the second interval duration and the preset data amount can be used in combination, for example
<dl ms="on",g="5400",r="5400",f=“20000”
<s t="aaaaaa-bbbbbb">
<s t="cccccc-dddddd">
<s t="eeeeee-ffffff">
</dl>
Wherein f represents a preset data amount, 20000 represents that the preset data amount which can be stored by the edge computing device is 20G, and if the data amount stored by the edge computing device is monitored to reach the preset data amount, even if a preset data acquisition time period still remains, the task data acquisition from the task server needs to be stopped.
S402: and generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data.
S403: and sending a data acquisition request instruction to the data source server, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from the storage area. One way is that: and sequentially sending data acquisition request instructions to the data source server based on the preset interval time. When the data acquisition request instruction is sent, responding to the data acquisition request instruction sent to the data source server, and recording data downloading time, wherein the data downloading time is the time consumed from the sending of the data acquisition request instruction to the receiving of the data to be distributed.
And if the data downloading time is less than or equal to the preset interval time, postponing a first waiting time after the moment of receiving the current data to be distributed to execute the step of sending a data acquisition request instruction to the data source server, wherein the first waiting time is the difference between the preset interval time and the data downloading time.
And if the data downloading time is greater than the preset interval time, delaying the preset second waiting time after the moment of receiving the current data to be distributed to execute the step of sending a data acquisition request instruction to the data source server.
For the control of sending the data obtaining request command, reference is made to the related description in the above method embodiment, and this embodiment is not described again.
S404: and receiving and storing the data to be distributed sent by the data source server.
According to the technical scheme, the edge computing device can store the data to be distributed in the storage area corresponding to the data source server, so that a service request for requesting the data to be distributed, which is sent to the data source server, can be responded by the edge computing device, and the edge computing device bears part of the service request of the data source server, so that the response speed of the service request is improved. And the peak value of the access flow for triggering the data source server to provide the service is reduced by adopting a mode that the edge computing equipment bears part of the service request of the data source server, the bandwidth cost is calculated on the basis of the peak value of the access flow, and when the peak value of the access flow is reduced, the bandwidth cost calculated according to the peak value of the access flow is also reduced.
The edge computing device can control the acquisition of the task data to control the edge computing device to send a data acquisition request instruction to the data source server through the control of the task data, so as to realize the control of the distribution speed of the data to be distributed, the edge computing device can also control the sending of two adjacent data acquisition request instructions, the edge computing device receives and stores the data to be distributed from the data source server at intervals of the sending of the two adjacent data acquisition request instructions, thereby realizing the control of the download speed of the data to be distributed through the control of the sending of the data acquisition request instructions, realizing the control of the total download data amount of the edge computing device through the control of the distribution speed and the download speed of the data to be distributed, and leading the total download data amount of the edge computing device in the download period to be close to the preset total amount, the total amount of the downloaded data of different edge computing devices in the same downloading period is the same, the total amount of the downloaded data of different edge computing devices can be similar through the control, which means that the services which can be provided by different edge computing devices are similar, the different edge computing devices can provide the similar services when requesting, and the load balance among different edge computing devices is realized.
In addition, in this embodiment, for each preset data obtaining time period, the total time consumed by any preset data obtaining time period from sending the first task data obtaining request instruction to ending storing the data to be distributed of each download task corresponding to the task data may be recorded, and if the time corresponding to the total time since sending the first task data obtaining request instruction is delayed exceeds the starting time of the next preset data obtaining time period, the task data obtaining request instruction is sent immediately.
For example, the plurality of preset data acquisition time periods for the edge computing device are represented as follows:
<dl<s t="aaaaaa-bbbbbb">
<s t="cccccc-dddddd">
<s t="eeeeee-ffffff">
</dl>
each T is a preset data acquisition time period, the edge computing device requests to acquire task data and stores data to be distributed in each preset data acquisition time period, if the total time consumed by acquiring the task data and storing the data to be distributed corresponding to each piece of downloaded data in T ═ aaaa-bbbb "is T, the edge computing device deduces the time corresponding to T from the time when acquiring the task data in T ═ aaaaaa-bbbbbb" exceeds the starting time when T ═ cccc-dddddd "starts, and then the edge computing device immediately sends a task data acquisition request instruction after finishing storing the data to be distributed corresponding to all the downloaded tasks in T ═ aaaaaa-bbbb".
That is, generally, the edge computing device can send the task data obtaining request instruction only within the preset data obtaining time period, but if the edge computing device stores the data to be distributed corresponding to the task data obtained in one preset data obtaining time period, even if the current time reaches the starting time of sending the task data obtaining request instruction to the task server in the next preset data obtaining time period, the edge computing device cannot send the task data obtaining request instruction to the task server, but needs the edge computing device to send the next task data obtaining request instruction to the task server immediately after completing the storage of the data to be distributed, so as to reduce the conflict between the task data obtaining and the data to be distributed received from the data source server, and realize the effective utilization of the processing resources of the edge computing device, the overall speed of the edge computing device in receiving data to be distributed from the data source server is improved.
The present embodiment may also add other parameters to the above-mentioned settings, for example, change as follows:
<dl i=“xxx”,ms="on",g="5400",r="5400",f=“20000”
<s t="aaaaaa-bbbbbb">
<s t="cccccc-dddddd">
<s t="eeeeee-ffffff">
</dl>
wherein i represents a network operator, such as mobile, internet, and telecommunications, and different network operators, and the value of i is different, for example, i ═ 1 indicates that the network operator is mobile, and i ═ 2 indicates that the network operator is internet. In this embodiment, i may also adopt a null value, that is, no data is configured for i, if i is a null value (such as i ═ xxx), it means that the edge computing device needs to adopt a default configuration, that is, the values of the parameters in the configuration expressed by dl are default values, if a certain network operator is found by the value of i, it needs to be determined first whether the edge computing device can use the network of the network operator, if the network of the network operator can be used, the configuration given by the network operator is used, that is, the values of the parameters in the configuration expressed by bl are set for the network operator corresponding to i, and if the network of the network operator cannot be used, the default configuration is used.
In addition, the data acquisition control method provided by the embodiment can also combine all the above settings, and the combined expression form is as follows:
<dl i=“xxx”,ms="on",g="5400",r="5400",f=“20000”
<s t="aaaaaa-bbbbbb",td=80>
<s t="cccccc-dddddd",td=70>
<s t="eeeeee-ffffff",td=75>
</dl>
different edge computing devices acquire task data from the task server according to respective corresponding settings, and receive data to be distributed from the data source server, so that the data volumes stored in different edge computing devices are the same or close to each other.
Referring to fig. 5, a data acquisition control method according to another embodiment of the present invention is shown, which is applied to a task server and includes the following steps:
s501: and generating task data. In this embodiment, the task server may configure task data according to the type of the edge computing device and the remaining storage capacity of the edge computing device, such as a task amount configured in a preset task data issuing time period, where the task amount is used to indicate the number of download tasks that the edge computing device can acquire, and one download task is used to trigger acquisition of data to be distributed at a time, and besides configuring the task amount, related parameters of the data source server corresponding to each task amount, such as an identifier of the data source server and an identifier of a service provided by the data source server, may also be configured to acquire data related to the service provided by the data source server from the configured data source server.
The form of the configuration task data is as follows:
<PR P="xxx">
<TR BT="01:00:00"ET="01:29:00"Count="30"/>
<TR BT="01:30:00"ET="01:59:00"Count="55"/>
</PR>
for a specific description, please refer to the above method embodiment, where BT, ET, and Count may all be configured according to actual situations, and the embodiment does not show the relevant parameters of the data source server corresponding to the task amount.
S502: and receiving a task data acquisition request instruction sent by the edge computing equipment.
S503: and responding to the task data acquisition request instruction, and sending task data to the edge computing equipment, wherein the task data is used for instructing the edge computing equipment to acquire the data to be distributed from the data source server.
For example, after receiving a task data acquisition request instruction, it is determined whether the time when the task data acquisition request instruction is received is within a preset task data delivery time period, and if so, the task data is sent to the edge computing device.
When sending task data to the edge computing device, the current amount of data to be stored (representing the current remaining storage amount of the edge computing device) of the edge computing device needs to be considered, for example, in response to a task data obtaining request instruction, one way of sending task data to the edge computing device is as follows:
extracting identification information of the edge computing equipment, the data volume to be stored of the edge computing equipment and the sending time of the task data acquisition request instruction from the task data acquisition request instruction; acquiring preset task data within a preset task data issuing time period corresponding to the sending time; and sending task data matched with the data amount to be stored to the edge computing device based on the identification information of the edge computing device and the data amount to be stored of the edge computing device.
The preset task data issuing time period is the time period corresponding to the BT and the ET, and since the task data allocated to each preset task data issuing time period may be different, after receiving the task data acquisition request instruction, the preset task data issuing time period corresponding to the sending time of the task data acquisition request instruction is determined, and then the task data matched with the data amount to be stored of the edge computing device is acquired from the task data of the corresponding preset task data issuing time period, so that the edge computing device can continuously store the data amount to be distributed of the data amount to be stored, the data amount of the data to be distributed is prevented from exceeding the data amount to be stored of the edge computing device, and the downloading of useless data (or redundant data) is reduced.
According to the technical scheme, the task server can control the sending of the task data, so that the control of sending the data acquisition request instruction to the data source server by the edge computing device is achieved through the control of the task data, and the control of the distribution speed of the data to be distributed is achieved.
An embodiment of the present invention further provides a data acquisition control device, which is applied to an edge computing device, and the structure of the data acquisition control device is shown in fig. 6, where the data acquisition control device may include: an acquisition unit 601, a generation unit 602, a transmission unit 603, and a reception storage unit 604.
The acquiring unit 601 is configured to acquire task data sent by a task server. One way is that: if the current moment of the edge computing equipment is within a preset data acquisition time period of the edge computing equipment, sending a task data acquisition request instruction to the task server; and receiving the task data sent by the task server.
The other mode is as follows: if the total data volume of the data to be distributed stored in the edge computing equipment is smaller than the preset data volume, acquiring task data sent by the task server; and if the total data volume of the data to be distributed stored in the edge computing equipment is larger than or equal to the preset data volume, forbidding to acquire the task data sent by the task server.
In addition, the data acquisition control apparatus provided in this embodiment further includes a determining unit, configured to determine, when the edge computing device runs for a first interval duration after the time when the task data is received, whether a current time corresponding to the first interval duration after the time when the task data is received is within a preset data acquisition time period of the edge computing device; if the current time is within a preset data acquisition time period of the edge computing device, triggering the acquisition unit 601 to send a task data acquisition request instruction to the task server; and if the current time is not within the preset data table acquisition time period of the edge computing device, prohibiting the acquisition unit 601 from sending a task data acquisition request instruction to the task server at the current time.
Or, the determining unit is configured to, if the task data is empty, determine, when the edge computing device operates for a second interval duration after a time when the received task data is empty, whether a current time corresponding to the second interval duration after the time when the received task data is empty is within a preset data obtaining time period of the edge computing device; if the data is in the preset data acquisition time period of the edge computing device, triggering the acquisition unit 601 to send a task data acquisition request instruction to the task server; and if the current time is not within the preset data acquisition time period of the edge computing device, postponing the current time backwards for a first interval duration to judge whether the time corresponding to the first interval duration postponed backwards from the current time is within the preset data acquisition time period of the edge computing device.
A generating unit 602, configured to generate a data obtaining request instruction based on a download task for obtaining data to be distributed in the task data.
A sending unit 603, configured to send a data obtaining request instruction to the data source server, where the data obtaining request instruction is used to trigger the data source server to obtain, from the storage area, data to be distributed that is matched with the data obtaining request instruction.
For example, the sending unit 603 sends data acquisition request instructions to the data source servers in sequence based on the preset interval time. In the process of sending a data acquisition request instruction to a data source server, responding to the data acquisition request instruction sent to the data source server, and recording data downloading time by a recording unit in a data acquisition control device, wherein the data downloading time is the time consumed from the sending of the data acquisition request instruction to the end of storing the data to be distributed; if the data download time is less than or equal to a preset interval time, the sending unit 603 postpones a first waiting time to send the data acquisition request instruction to a data source server after the time of finishing storing the data to be distributed, wherein the first waiting time is a difference between the preset interval time and the data download time; if the data downloading time is greater than the preset interval time, the sending unit 603 postpones a preset second waiting time to send the data obtaining request instruction to the data source server after the time of finishing storing the data to be distributed.
The receiving and storing unit 604 is configured to receive and store data to be distributed sent by the data source server.
Referring to fig. 7, it is shown that another data acquisition control device provided in an embodiment of the present invention is applied to a task server, and may include: a generating unit 701, a receiving unit 702 and a responding unit 703.
A generating unit 701 configured to generate task data. In this embodiment, the generating unit 701 may configure task data according to the type of the edge computing device and the remaining storage capacity of the edge computing device, such as a task amount configured in a preset task data delivery time period, where the task amount is used to indicate the number of download tasks that can be acquired by the edge computing device, and one download task is used to trigger acquisition of data to be distributed at a time, and may also configure relevant parameters of the data source server corresponding to each task amount, such as an identifier of the data source server and an identifier of a service provided by the data source server, in addition to the configured task amount, so as to acquire data related to the service provided by the data source server from the configured data source server.
The receiving unit 702 is configured to receive a task data obtaining request instruction sent by an edge computing device.
A responding unit 703, configured to send, in response to a task data obtaining request instruction, the task data to the edge computing device, where the task data is used to instruct the edge computing device to obtain data to be distributed from the data source server. For example, after receiving a task data acquisition request instruction, it is determined whether the time when the task data acquisition request instruction is received is within a preset task data delivery time period, and if so, the task data is sent to the edge computing device.
When sending task data to the edge computing device, the current amount of data to be stored (representing the current remaining storage amount of the edge computing device) of the edge computing device needs to be considered, for example, in response to a task data obtaining request instruction, one way of sending task data to the edge computing device is as follows:
extracting identification information of the edge computing equipment, the data volume to be stored of the edge computing equipment and the sending time of the task data acquisition request instruction from the task data acquisition request instruction; acquiring preset task data within a preset task data issuing time period corresponding to the sending time; and sending task data matched with the data amount to be stored to the edge computing device based on the identification information of the edge computing device and the data amount to be stored of the edge computing device.
The preset task data issuing time period is the time period corresponding to the BT and the ET, and since the task data allocated to each preset task data issuing time period may be different, after receiving the task data acquisition request instruction, the preset task data issuing time period corresponding to the sending time of the task data acquisition request instruction is determined, and then the task data matched with the data amount to be stored of the edge computing device is acquired from the task data of the corresponding preset task data issuing time period, so that the edge computing device can continuously store the data amount to be distributed of the data amount to be stored, the data amount of the data to be distributed is prevented from exceeding the data amount to be stored of the edge computing device, and the downloading of useless data (or redundant data) is reduced.
The embodiment of the invention also provides edge computing equipment, which comprises a processor, a memory and a communication interface.
The processor is used for acquiring task data sent by the task server, generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data, and sending the data acquisition request instruction to the data source server through the communication interface, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from the storage area; and the processor is also used for receiving the data to be distributed sent by the data source server through the communication interface and storing the data to be distributed into the memory. For the execution process of the processor, reference may be made to the data acquisition control method shown in fig. 4, which is not described in this embodiment.
An embodiment of the present invention further provides a task server, where the task server includes: a processor and a communication interface.
The processor is used for generating task data and receiving a task data acquisition request instruction sent by the edge computing equipment through the communication interface; and the processor is also used for responding to the task data acquisition request instruction and sending task data to the edge computing equipment through the communication interface, wherein the task data is used for instructing the edge computing equipment to acquire the data to be distributed from the data source server. For the execution process of the processor in the task server, reference may be made to the data acquisition control method shown in fig. 5, which is not described in this embodiment.
The embodiment of the invention also provides a storage medium, wherein the storage medium stores a program code, and the program code realizes the data acquisition control method when being operated by the processor.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (17)

1. A data acquisition control system, characterized in that the system comprises: an edge computing device and a task server;
the edge computing device is used for sending a task data acquisition request instruction to the task server;
the task server is used for responding to the task data acquisition request instruction and sending the task data generated by the task server to the edge computing equipment so as to control the edge computing equipment to acquire data from a data source server; the task data is used for indicating the edge computing equipment to acquire a source of the data and a service corresponding to the data;
the edge computing device is further configured to generate a data obtaining request instruction based on a download task used for obtaining data to be distributed in the task data, send the data obtaining request instruction to a data source server, where the data obtaining request instruction is used to trigger the data source server to obtain the data to be distributed matched with the data obtaining request instruction from a storage area, and receive and store the data to be distributed sent by the data source server.
2. An edge computing device, the edge computing device comprising: a processor, a memory, and a communication interface;
the processor is used for acquiring task data sent by the task server so as to control the edge computing equipment to acquire data from the data source server; the task data is used for indicating the edge computing equipment to acquire a source of the data and a service corresponding to the data; generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data, and sending the data acquisition request instruction to a data source server through the communication interface, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area;
the processor is further configured to receive the data to be distributed sent by the data source server through the communication interface, and store the data to be distributed in the memory.
3. A task server, wherein the task server comprises: a processor and a communication interface;
the processor is used for generating task data and receiving a task data acquisition request instruction sent by the edge computing equipment through the communication interface;
the processor is further configured to send the task data to the edge computing device through the communication interface in response to the task data obtaining request instruction, where the task data is used to indicate a source of the obtained data and a service corresponding to the data to control the edge computing device to obtain the data to be distributed from a data source server.
4. A data acquisition control method, characterized in that the method comprises:
the edge computing equipment sends a task data acquisition request instruction to the task server;
the task server responds to the task data acquisition request instruction and sends the task data generated by the task server to the edge computing equipment so as to control the edge computing equipment to acquire data from a data source server; the task data is used for indicating the edge computing equipment to acquire a source of the data and a service corresponding to the data;
the edge computing equipment generates a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data;
the edge computing device sends the data acquisition request instruction to a data source server, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area;
and the edge computing equipment receives and stores the data to be distributed sent by the data source server.
5. A data acquisition control method, applied to an edge computing device, includes:
acquiring task data sent by a task server to control the edge computing equipment to acquire data from a data source server; the task data is used for indicating the edge computing equipment to acquire a source of the data and a service corresponding to the data;
generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data;
sending the data acquisition request instruction to a data source server, wherein the data acquisition request instruction is used for triggering the data source server to acquire the data to be distributed matched with the data acquisition request instruction from a storage area;
and receiving and storing the data to be distributed sent by the data source server.
6. The method of claim 5, wherein sending the data fetch request instruction to a data origin server comprises:
and sequentially sending data acquisition request instructions to the data source server based on preset interval time.
7. The method of claim 5 or 6, wherein in response to sending the data fetch request instruction to a data origin server, the method further comprises:
recording data downloading time, wherein the data downloading time is the time consumed from the time of sending the data acquisition request instruction to the time of finishing storing the data to be distributed;
if the data downloading time is less than or equal to a preset interval time, postponing a first waiting time after finishing storing the data to be distributed to execute the step of sending the data acquisition request instruction to a data source server, wherein the first waiting time is the difference between the preset interval time and the data downloading time;
and if the data downloading time is greater than the preset interval time, delaying a preset second waiting time after the moment of storing the data to be distributed is finished, and executing the step of sending the data acquisition request instruction to the data source server.
8. The method of claim 5, wherein the obtaining task data sent by a task server comprises:
if the current moment of the edge computing equipment is within a preset data acquisition time period of the edge computing equipment, sending a task data acquisition request instruction to the task server;
and receiving the task data sent by the task server.
9. The method of claim 8, wherein after receiving the task data sent by the task server, the method further comprises:
when the edge computing device runs for a first interval time after the moment of receiving the task data, judging whether the current moment corresponding to the first interval time after the moment of receiving the task data runs is within a preset data acquisition time period of the edge computing device;
if the current moment is within a preset data acquisition time period of the edge computing device, executing a step of sending a task data acquisition request instruction to the task server;
and if the current time is not within the preset data table acquisition time period of the edge computing equipment, forbidding sending a task data acquisition request instruction to the task server at the current time.
10. The method of claim 8, wherein after receiving the task data sent by the task server, the method further comprises:
if the task data is empty, when the edge computing device operates a second interval duration after the time when the received task data is empty, judging whether the current time corresponding to the second interval duration after the time when the received task data is empty is within a preset data acquisition time period of the edge computing device;
if the task data are within the preset data acquisition time period of the edge computing equipment, a step of sending a task data acquisition request instruction to the task server is executed;
and if the current time is not within the preset data acquisition time period of the edge computing device, postponing the current time backwards for a first interval duration to judge whether the time corresponding to the first interval duration postponed backwards from the current time is within the preset data acquisition time period of the edge computing device.
11. The method of claim 5, wherein the obtaining task data sent by a task server comprises: and if the total data volume of the data to be distributed stored in the edge computing equipment is less than the preset data volume, acquiring the task data sent by the task server.
12. The method of claim 5, further comprising: and if the total data volume of the data to be distributed stored in the edge computing equipment is larger than or equal to the preset data volume, forbidding to acquire the task data sent by the task server.
13. A data acquisition control method is applied to a task server and comprises the following steps:
generating task data;
receiving a task data acquisition request instruction sent by edge computing equipment;
and responding to the task data acquisition request instruction, and sending the task data to the edge computing equipment, wherein the task data is used for indicating the source of the data acquired by the edge computing equipment and the service corresponding to the data so as to control the edge computing equipment to acquire the data to be distributed from a data source server.
14. The method of claim 13, wherein sending the task data to the edge computing device in response to the task data fetch request instruction comprises:
responding to the task data acquisition request instruction, and if the moment of receiving the task data acquisition request instruction is within a preset task data issuing time period, sending the task data to the edge computing equipment.
15. The method of claim 13, wherein sending the task data to the edge computing device in response to the task data fetch request instruction comprises:
extracting identification information of the edge computing device, the data volume to be stored of the edge computing device and the sending time of the task data acquisition request instruction from the task data acquisition request instruction;
acquiring preset task data within a preset task data issuing time period corresponding to the sending time;
and sending task data matched with the data volume to be stored to the edge computing equipment based on the identification information of the edge computing equipment and the data volume to be stored of the edge computing equipment.
16. A data acquisition control apparatus, applied to an edge computing device, comprising:
the acquisition unit is used for acquiring task data sent by a task server so as to control the edge computing equipment to acquire the data from a data source server; the task data is used for indicating the edge computing equipment to acquire a source of the data and a service corresponding to the data;
the generating unit is used for generating a data acquisition request instruction based on a downloading task for acquiring data to be distributed in the task data;
a sending unit, configured to send the data obtaining request instruction to a data source server, where the data obtaining request instruction is used to trigger the data source server to obtain, from a storage area, the data to be distributed that is matched with the data obtaining request instruction;
and the receiving and storing unit is used for receiving and storing the data to be distributed sent by the data source server.
17. A data acquisition control apparatus, applied to a task server, comprising:
a generation unit configured to generate task data;
the receiving unit is used for receiving a task data acquisition request instruction sent by the edge computing equipment;
and the response unit is used for responding to the task data acquisition request instruction and sending the task data to the edge computing equipment, wherein the task data is used for indicating the source of the data acquired by the edge computing equipment and the service corresponding to the data so as to control the edge computing equipment to acquire the data to be distributed from the data source server.
CN201910911297.XA 2019-09-25 2019-09-25 Data acquisition control system, data acquisition control method and related device Active CN110691127B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910911297.XA CN110691127B (en) 2019-09-25 2019-09-25 Data acquisition control system, data acquisition control method and related device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910911297.XA CN110691127B (en) 2019-09-25 2019-09-25 Data acquisition control system, data acquisition control method and related device

Publications (2)

Publication Number Publication Date
CN110691127A CN110691127A (en) 2020-01-14
CN110691127B true CN110691127B (en) 2022-04-12

Family

ID=69110153

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910911297.XA Active CN110691127B (en) 2019-09-25 2019-09-25 Data acquisition control system, data acquisition control method and related device

Country Status (1)

Country Link
CN (1) CN110691127B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112202896A (en) * 2020-09-30 2021-01-08 中移(杭州)信息技术有限公司 Edge calculation method, frame, terminal and storage medium
CN112272239B (en) * 2020-10-30 2022-09-02 深圳市雷鸟网络传媒有限公司 Edge calculation method and device, intelligent terminal and computer readable storage medium
CN112486667B (en) * 2020-11-03 2022-03-18 深圳市中博科创信息技术有限公司 Method and device for accurately processing data based on edge calculation
CN112486676B (en) * 2020-11-25 2023-04-18 深圳市中博科创信息技术有限公司 Data sharing and distributing device based on edge calculation
CN115361032B (en) * 2022-08-17 2023-04-18 佛山市朗盛通讯设备有限公司 Antenna unit for 5G communication

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075954A (en) * 2006-06-12 2007-11-21 腾讯科技(深圳)有限公司 Point-to-point fluid-medium telecommunication system and method for allocating supernode resources
CN101150421A (en) * 2006-09-22 2008-03-26 华为技术有限公司 A distributed content distribution method, edge server and content distribution network
CN101626385A (en) * 2009-08-10 2010-01-13 中兴通讯股份有限公司 Media service method and media service system
EP2523423A1 (en) * 2011-05-10 2012-11-14 Deutsche Telekom AG Method and system for providing a distributed scalable hosting environment for web services
CN104253838A (en) * 2013-06-27 2014-12-31 北京世纪互联宽带数据中心有限公司 CDN (content delivery network) node and CDN service system
CN108121511A (en) * 2017-12-21 2018-06-05 北京黑螺技术有限公司 Data processing method, device and equipment in a kind of distributed edge storage system
CN108683730A (en) * 2018-05-11 2018-10-19 深圳市网心科技有限公司 Resource regulating method, service server, client terminal, network system and medium
CN108810170A (en) * 2018-07-19 2018-11-13 中国联合网络通信集团有限公司 resource allocation method and system
CN108984433A (en) * 2017-06-05 2018-12-11 华为技术有限公司 Cache data control method and equipment
CN109194984A (en) * 2018-11-09 2019-01-11 中山大学 A kind of video frame dispatching method based on edge calculations
CN109274745A (en) * 2018-09-28 2019-01-25 北京北斗方圆电子科技有限公司 A kind of Internet of things system and method for fringe node optimization calculating
CN110198339A (en) * 2019-04-17 2019-09-03 浙江大学 A kind of edge calculations method for scheduling task based on QoE perception
CN110213327A (en) * 2019-04-23 2019-09-06 广东省智能制造研究所 A kind of resource regulating method based on edge calculations, apparatus and system
CN110247977A (en) * 2019-06-17 2019-09-17 中国联合网络通信集团有限公司 A kind of method and system of the data fusion based on edge calculations

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6825296B2 (en) * 2016-10-11 2021-02-03 富士通株式会社 Edge server and its encrypted communication control method
US11093287B2 (en) * 2019-05-24 2021-08-17 Intel Corporation Data management for edge architectures

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101075954A (en) * 2006-06-12 2007-11-21 腾讯科技(深圳)有限公司 Point-to-point fluid-medium telecommunication system and method for allocating supernode resources
CN101150421A (en) * 2006-09-22 2008-03-26 华为技术有限公司 A distributed content distribution method, edge server and content distribution network
CN101626385A (en) * 2009-08-10 2010-01-13 中兴通讯股份有限公司 Media service method and media service system
EP2523423A1 (en) * 2011-05-10 2012-11-14 Deutsche Telekom AG Method and system for providing a distributed scalable hosting environment for web services
CN104253838A (en) * 2013-06-27 2014-12-31 北京世纪互联宽带数据中心有限公司 CDN (content delivery network) node and CDN service system
CN108984433A (en) * 2017-06-05 2018-12-11 华为技术有限公司 Cache data control method and equipment
CN108121511A (en) * 2017-12-21 2018-06-05 北京黑螺技术有限公司 Data processing method, device and equipment in a kind of distributed edge storage system
CN108683730A (en) * 2018-05-11 2018-10-19 深圳市网心科技有限公司 Resource regulating method, service server, client terminal, network system and medium
CN108810170A (en) * 2018-07-19 2018-11-13 中国联合网络通信集团有限公司 resource allocation method and system
CN109274745A (en) * 2018-09-28 2019-01-25 北京北斗方圆电子科技有限公司 A kind of Internet of things system and method for fringe node optimization calculating
CN109194984A (en) * 2018-11-09 2019-01-11 中山大学 A kind of video frame dispatching method based on edge calculations
CN110198339A (en) * 2019-04-17 2019-09-03 浙江大学 A kind of edge calculations method for scheduling task based on QoE perception
CN110213327A (en) * 2019-04-23 2019-09-06 广东省智能制造研究所 A kind of resource regulating method based on edge calculations, apparatus and system
CN110247977A (en) * 2019-06-17 2019-09-17 中国联合网络通信集团有限公司 A kind of method and system of the data fusion based on edge calculations

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Social-Aware Edge Caching in Fog Radio Access Networks;Xiang Wang et al.;《IEEE Access》;20170424;全文 *
面向边缘计算的资源优化技术研究进展;屈志昊 等;《大数据 》;20190308;全文 *

Also Published As

Publication number Publication date
CN110691127A (en) 2020-01-14

Similar Documents

Publication Publication Date Title
CN110691127B (en) Data acquisition control system, data acquisition control method and related device
CN109862065B (en) File downloading method and device and electronic equipment
US8935692B2 (en) Self-management of virtual machines in cloud-based networks
US9870541B2 (en) Service level backup using re-cloud network
CN106302434B (en) Server adaptation method, device and system
CN107861760A (en) BIOS collocation method, terminal and server
CN102045321B (en) Flow control method and system for software downloading
CN108667938B (en) Traffic switching method and device and server
CN106897299B (en) Database access method and device
CN110995617B (en) MQTT-based data transmission method and device, computer equipment and storage medium
CN107948097B (en) Bandwidth adjusting method and equipment
WO2010054396A4 (en) Method and apparatus for managing software downloads in a broadcast communication system
CN102413514B (en) Data distribution device and data distribution system
CN111177160B (en) Service updating method, device, server and medium
CN107908730B (en) Method and device for downloading data
CN109831474B (en) Keep-alive system, method, server and readable storage medium for http long connection
CN111064700B (en) Downloading method, device and system of cloud game
CN108040115B (en) File transmission method and device
CN106790632B (en) Streaming data concurrent transmission method and device
CN113660117B (en) Method and device for accelerating content delivery of terminal equipment and content delivery system
CN101917458A (en) Method and device for sending network bootstrap program to client
CN109831467A (en) Data transmission method, equipment and system
EP2159985A1 (en) Method, apparatus and system for scheduling contents
CN113872871A (en) Network acceleration method and system
CN111726658A (en) Multimedia data transmission method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant