CN112737896B

CN112737896B - Bandwidth data checking method, device, medium and electronic equipment

Info

Publication number: CN112737896B
Application number: CN202110356211.9A
Authority: CN
Inventors: 杨立坤
Original assignee: Beijing Horizon Yuntian Technology Co ltd
Current assignee: Shenzhen Shijie Yuntian Technology Co ltd
Priority date: 2021-04-01
Filing date: 2021-04-01
Publication date: 2021-06-25
Anticipated expiration: 2041-04-01
Also published as: CN112737896A

Abstract

The disclosure relates to a method, an apparatus, a medium, and an electronic device for checking bandwidth data. The method applied to the scheduling node comprises the following steps: acquiring a plurality of complementary tasks popped from a task queue, grouping according to the incidence relation among the plurality of complementary tasks to obtain at least one complementary task group, generating a complementary task when an acquisition check node fails to check bandwidth data, sending the complementary task to a scheduling node, and pressing the complementary task into the task queue by the scheduling node; aiming at each complementary acquisition task group, determining target nodes for executing all complementary acquisition tasks in the complementary acquisition task group from a plurality of acquisition check nodes; and sending each complementary acquisition task group and each target node to a task queue node. Therefore, each complementary acquisition task with the association relation in one complementary acquisition task group is intensively executed by the same acquisition checking node, the execution time difference of each complementary acquisition task with the association relation is short, the bandwidth data related to the complementary acquisition task group can be checked again in time, and the checking efficiency of the bandwidth data is ensured.

Description

Bandwidth data checking method, device, medium and electronic equipment

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a method, an apparatus, a medium, and an electronic device for checking bandwidth data.

Background

In the multi-cloud bandwidth data acquisition process, bandwidth data needs to be acquired from each bandwidth service provider (namely, a cloud manufacturer), and the experience of bandwidth service users is directly influenced if the bandwidth data is accurate and real-time. During the bandwidth data acquisition process, problems such as interface faults, network faults and the like may occur, which will affect the timeliness of the bandwidth data acquisition and the accuracy of the bandwidth data. Therefore, it is necessary to check the acquired bandwidth data, and when the check fails, the bandwidth data is acquired again, and then, the check is performed again based on the acquired data. How to ensure the high-efficiency execution of the complementary acquisition task is the key for improving the checking efficiency of the bandwidth data.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a bandwidth data checking method, apparatus, medium, and electronic device.

In order to achieve the above object, in a first aspect, the present disclosure provides a method for checking bandwidth data, which is applied to a scheduling node in a bandwidth data checking system, where the bandwidth data checking system includes the scheduling node, a task queue node, and a plurality of acquisition checking nodes, and the method includes:

acquiring a plurality of complementary tasks popped up from a task queue, grouping the plurality of complementary tasks according to the incidence relation among the plurality of complementary tasks to obtain at least one complementary task group, wherein the complementary tasks are used for indicating to acquire offline logs or bandwidth data again, generating the complementary tasks when the acquisition check nodes determine that the bandwidth data check is failed, and sending the complementary tasks to the scheduling nodes so that the scheduling nodes press the complementary tasks into the task queue;

for each complementary acquisition task group, determining a target node for executing all complementary acquisition tasks in the complementary acquisition task group from the plurality of acquisition check nodes;

and sending each complementary acquisition task group and target nodes for executing all complementary acquisition tasks in each complementary acquisition task group to the task queue nodes.

Optionally, the complementary task includes a complementary period and a bandwidth service user;

the grouping the plurality of complementary tasks according to the incidence relation among the plurality of complementary tasks comprises:

and taking the complementary tasks with the same bandwidth service users and the same complementary acquisition time interval as a complementary acquisition task group.

In a second aspect, the present disclosure provides a method for checking bandwidth data, which is applied to any acquisition checking node in a bandwidth data checking system, where the bandwidth data checking system includes a scheduling node, a task queue node, and a plurality of the acquisition checking nodes, and the method includes:

responding to real-time bandwidth data of a bandwidth interface of any bandwidth service provider acquired at the current acquisition time, and acquiring an offline log of the bandwidth service provider and historical bandwidth data of the bandwidth interface acquired at the previous acquisition time of the current acquisition time in a period of time to which the current acquisition time belongs;

performing bandwidth data check according to the offline log, the historical bandwidth data and the real-time bandwidth data;

and if the bandwidth data check is not passed, generating a complementary acquisition task for indicating to acquire the real-time bandwidth data of the bandwidth service provider or the bandwidth interface at the current acquisition time within the time period to which the current acquisition time belongs, and sending the complementary acquisition task to the scheduling node so that the scheduling node presses the complementary acquisition task into a task queue.

Optionally, the method further comprises:

determining the comparable bandwidth data corresponding to the historical contemporary moment of the bandwidth interface at the current acquisition moment;

the checking the bandwidth data according to the offline log, the historical bandwidth data and the real-time bandwidth data comprises:

and performing bandwidth data check according to the offline log, the historical bandwidth data, the real-time bandwidth data and the comparable bandwidth data.

Optionally, the determining comparable bandwidth data corresponding to the historical contemporaneous time of the bandwidth interface at the current acquisition time includes:

acquiring bandwidth data of the bandwidth interface, which is acquired at each historical acquisition time within the date of the historical synchronization time;

generating a fitting curve of the acquisition time and the bandwidth data according to the bandwidth data of the bandwidth interface acquired at each historical acquisition time;

according to the fitting curve, determining fitting bandwidth data corresponding to the bandwidth interface at the historical synchronization moment and fitting bandwidth data corresponding to the bandwidth interface at a plurality of adjacent acquisition moments of the historical synchronization moment;

and determining the average value of all the fitting bandwidth data as the comparable bandwidth data corresponding to the bandwidth interface at the historical synchronization moment.

Optionally, the performing bandwidth data checking according to the offline log, the historical bandwidth data, the real-time bandwidth data, and the comparable bandwidth data includes:

determining the offline bandwidth data of the bandwidth interface at the current acquisition moment according to the offline log;

determining that the bandwidth data check fails if any of the following conditions are met:

the offline bandwidth data and the real-time bandwidth data meet a first preset condition;

the historical bandwidth data and the real-time bandwidth data meet a second preset condition;

and the comparable bandwidth data and the real-time bandwidth data meet a third preset condition.

Optionally, if the checking of the bandwidth data fails, generating a complementary acquisition task for instructing to reacquire the bandwidth data of the bandwidth service provider or the bandwidth interface at the current acquisition time within the time period to which the current acquisition time belongs, including:

if the offline bandwidth data and the real-time bandwidth data meet the first preset condition and the offline bandwidth data are smaller than the real-time bandwidth data, generating a complementary collection task for indicating to collect the offline logs of the bandwidth service provider in the time period to which the current collection time belongs again;

generating a complementary acquisition task for instructing to reacquire real-time bandwidth data of the bandwidth interface at the current acquisition time if any one of the following conditions is met:

the historical bandwidth data and the real-time bandwidth data meet the second preset condition;

the geometric bandwidth data and the real-time bandwidth data meet the third preset condition;

the offline bandwidth data and the real-time bandwidth data meet the first preset condition, and the offline bandwidth data is larger than the real-time bandwidth data.

In a third aspect, the present disclosure provides a bandwidth data checking apparatus, applied to a scheduling node in a bandwidth data checking system, where the bandwidth data checking system includes the scheduling node, a task queue node, and a plurality of acquisition checking nodes, and the apparatus includes:

the grouping module is used for acquiring a plurality of complementary tasks popped up from a task queue, grouping the plurality of complementary tasks according to the incidence relation among the plurality of complementary tasks to obtain at least one complementary task group, wherein the complementary tasks are used for indicating to re-collect offline logs or bandwidth data, generating the complementary tasks when the collection checking nodes determine that the bandwidth data cannot be checked, and sending the complementary tasks to the scheduling nodes so that the scheduling nodes press the complementary tasks into the task queue;

a determining module, configured to determine, for each complementary acquisition task group obtained by the grouping module, a target node for executing all complementary acquisition tasks in the complementary acquisition task group from the plurality of acquisition check nodes;

and the sending module is used for sending each complementary acquisition task group obtained by the grouping module and the target nodes which are determined by the determining module and used for executing all complementary acquisition tasks in each complementary acquisition task group to the task queue nodes.

In a fourth aspect, the present disclosure provides a bandwidth data checking apparatus, which is applied to any acquisition checking node in a bandwidth data checking system, where the bandwidth data checking system includes a scheduling node, a task queue node, and a plurality of the acquisition checking nodes, and the apparatus includes:

the acquisition module is used for responding to real-time bandwidth data of a bandwidth interface of any bandwidth service provider acquired at the current acquisition time, and acquiring an offline log of the bandwidth service provider and historical bandwidth data of the bandwidth interface acquired at the previous acquisition time of the current acquisition time in a period of time to which the current acquisition time belongs;

the checking module is used for checking the bandwidth data according to the offline log, the historical bandwidth data and the real-time bandwidth data acquired by the acquiring module;

and the generating module is used for generating a complementary acquisition task for indicating to acquire the offline log of the bandwidth service provider or the real-time bandwidth data of the bandwidth interface at the current acquisition time again in the time period of the current acquisition time if the checking module determines that the bandwidth data is not checked, and sending the complementary acquisition task to the scheduling node so that the scheduling node presses the complementary acquisition task into a task queue.

In a fifth aspect, the present disclosure provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method provided by the first or second aspect of the present disclosure.

In a sixth aspect, the present disclosure provides an electronic device comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method provided by the first aspect of the present disclosure.

In a seventh aspect, the present disclosure provides an electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method provided by the second aspect of the present disclosure.

In the technical scheme, the acquisition check node responds to real-time bandwidth data of a bandwidth interface of any bandwidth service provider acquired at the current acquisition time, and acquires an offline log of the bandwidth service provider and historical bandwidth data of the bandwidth interface acquired at the previous acquisition time of the current acquisition time within a period of time to which the current acquisition time belongs; then, according to the offline log, the historical bandwidth data and the real-time bandwidth data, performing bandwidth data check; and if the bandwidth data check is not passed, generating a complementary acquisition task for indicating to acquire the offline log of the bandwidth service provider or the real-time bandwidth data of the bandwidth interface at the current acquisition time again in the period of time to which the current acquisition time belongs, and sending the complementary acquisition task to a scheduling node so that the scheduling node presses the complementary acquisition task into a task queue. When the acquisition check node checks the bandwidth data, the acquisition check node not only refers to the offline log, but also considers the historical bandwidth data, so that the timeliness and the accuracy of the bandwidth data check can be ensured, and the bandwidth data check is not required to be performed manually, so that the time and the labor are saved. In addition, after acquiring a plurality of complementary acquisition tasks popped from the task queue, the scheduling node groups the complementary acquisition tasks to be executed according to the incidence relation among the plurality of complementary acquisition tasks to obtain at least one complementary acquisition task group; next, aiming at each complementary acquisition task group, determining target nodes for executing all complementary acquisition tasks in the complementary acquisition task group from a plurality of acquisition check nodes; and finally, sending each complementary acquisition task group and target nodes for executing all complementary acquisition tasks in each complementary acquisition task group to the task queue nodes. Therefore, each complementary acquisition task with the association relation in one complementary acquisition task group is intensively executed by the same acquisition checking node, so that the execution time difference of each complementary acquisition task with the association relation is short, the bandwidth data related to the complementary acquisition task group can be checked again in time, and the checking efficiency of the bandwidth data is ensured.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:

FIG. 1 is a block diagram illustrating a bandwidth data reconciliation system in accordance with an exemplary embodiment.

Fig. 2 is a flowchart illustrating a bandwidth data checking method applied to a scheduling node according to an exemplary embodiment.

FIG. 3 is a flow diagram illustrating a bandwidth data reconciliation method applied to an acquisition reconciliation node in accordance with an exemplary embodiment.

Fig. 4 is a block diagram illustrating a bandwidth data checking apparatus applied to a scheduling node according to an exemplary embodiment.

FIG. 5 is a block diagram illustrating a bandwidth data reconciliation apparatus applied to an acquisition reconciliation node in accordance with an exemplary embodiment.

FIG. 6 is a block diagram illustrating an electronic device in accordance with an example embodiment.

FIG. 7 is a block diagram illustrating an electronic device in accordance with an example embodiment.

FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.

The present disclosure provides a bandwidth data checking system, as shown in fig. 1, the bandwidth data checking system includes a scheduling node, a task queue node, and a plurality of acquisition checking nodes, where the scheduling node is connected to each acquisition checking node and the task queue node, and the task queue node is connected to each acquisition checking node.

Each acquisition check node is used for counting the bandwidth data of a plurality of bandwidth service users in charge of the acquisition check node, wherein one bandwidth service user can use the bandwidth service provided by a plurality of bandwidth service providers in a period of time. Specifically, for each bandwidth service user, the acquisition check node is configured to acquire bandwidth data (i.e., real-time bandwidth data) from a bandwidth interface, used by the bandwidth service user and provided by each bandwidth service provider, at a preset time interval (e.g., 5 minutes), and download an offline log from a corresponding log download interface, and at the same time, acquire historical bandwidth data, e.g., historical bandwidth data at a previous acquisition time; then, according to the offline log, the historical bandwidth data and the real-time bandwidth data, performing bandwidth data check; if the bandwidth data check is not passed, it is indicated that the real-time bandwidth data or the offline bandwidth data at the corresponding moment in the offline log may have a problem, at this time, an additional acquisition task for indicating to acquire the offline log or the bandwidth data again is generated, and the additional acquisition task is sent to the scheduling node, so that the scheduling node presses the additional acquisition task into a task queue; and if the bandwidth data check is passed, the real-time bandwidth data is accurate, and at the moment, the complementary acquisition task is not generated. The replenishment task may include a replenishment period, a bandwidth service user, a bandwidth service provider, a replenishment type (replenishment offline log, replenishment bandwidth data), a replenishment state (e.g., not started, during replenishment, completion of replenishment, failure of replenishment, etc.), a repeatable number of replenishment failures (e.g., 3 times), and the like.

After receiving a complementary acquisition task sent by any acquisition check node, the scheduling node presses the complementary acquisition task into a task queue; the scheduling node can take out a plurality of complementary tasks from the task queue at one time according to a preset time interval, wherein the number of the complementary tasks taken out each time can be the same or different, and then the plurality of complementary tasks are grouped to obtain at least one complementary task group; aiming at each complementary acquisition task group, determining target nodes for executing all complementary acquisition tasks in the complementary acquisition task group from a plurality of acquisition check nodes in the system; and finally, sending each complementary acquisition task group and target nodes for executing all complementary acquisition tasks in each complementary acquisition task group to the task queue nodes.

After the task queue node receives at least one complementary acquisition task group sent by the scheduling node and the target nodes for executing all complementary acquisition tasks in each complementary acquisition task group, the identifiers of the target nodes for executing all complementary acquisition tasks in the complementary acquisition task group and the complementary acquisition task group can be pressed into the task group queue for each complementary acquisition task group. In addition, each acquisition check node defines the task group queue theme, so that each acquisition check node can receive a complementary acquisition task issued by a scheduling node through a task queue node.

Illustratively, the task queue node may be a node deployed with kafka, which is a high-throughput distributed publish-subscribe message system, and is capable of providing high throughput for publish and subscribe simultaneously, and also performing persistence operation.

In addition, each acquisition check node is also used for being responsible for executing each complementary acquisition task in the complementary acquisition task group. Specifically, the acquisition check node acquires a complementary acquisition task group from a task group queue in the task queue node; judging whether the complementary acquisition task group is a complementary acquisition task group to be executed by the node according to the target node identification corresponding to the complementary acquisition task group; if yes, pressing the complementary mining task into a complementary mining queue corresponding to the node by the complementary mining core to wait or immediately execute; if not, the complementary task group is discarded.

And when the acquisition check node executes each complementary acquisition task in the complementary acquisition task group in the complementary acquisition queue, different interfaces are called according to the complementary acquisition type of the complementary acquisition task so as to carry out complementary acquisition. Specifically, if the complementary mining type is a complementary mining offline log, a log downloading interface provided by a bandwidth service provider in the complementary mining task is called to download the offline log so as to perform complementary mining of the offline log; and if the complementary acquisition type is complementary acquisition bandwidth data, calling a bandwidth interface provided by a bandwidth service provider in the complementary acquisition task to acquire the bandwidth data at the corresponding moment again so as to perform complementary acquisition of the bandwidth data. After the supplementary mining is finished, performing bandwidth data check again according to the data after supplementary mining; if the bandwidth data passes the check, the complementary mining is completed; and if the bandwidth data check is not passed, performing the complementary mining again until the complementary mining times reach the repeated times of the complementary mining failure corresponding to the complementary mining task.

Specifically, the scheduling node may perform the scheduling of the complementary tasks through S201 to S203 shown in fig. 2.

In S201, multiple complementary and mining tasks popped up from the task queue are obtained, and the multiple complementary and mining tasks are grouped according to the association relationship between the multiple complementary and mining tasks, so as to obtain at least one complementary and mining task group.

In the disclosure, the complementary acquisition task is used for indicating to acquire offline logs or bandwidth data again, and when the acquisition check node determines that the bandwidth data check is failed, the complementary acquisition task is generated and sent to the scheduling node so as to be pressed into a task queue by the scheduling node.

In S202, for each complementary acquisition task group, a target node for executing all complementary acquisition tasks in the complementary acquisition task group is determined from the plurality of acquisition core nodes.

In S203, each complementary task group and a target node for executing all complementary tasks in each complementary task group are sent to a task queue node.

A specific embodiment of grouping the plurality of complementary tasks according to the association relationship between the plurality of complementary tasks in S201 will be described in detail below. Specifically, the complementary tasks include complementary time periods and bandwidth service users, and in one embodiment, a complementary task that is the same as a bandwidth service user in the acquired plurality of complementary tasks may be used as a complementary task group. Therefore, each complementary task belonging to the same bandwidth service user can be executed in a centralized manner, so that the execution time difference is short, the bandwidth data of the bandwidth service user can be checked again in time, and the checking efficiency of the bandwidth data of the bandwidth service user is ensured.

In another embodiment, the complementary tasks with the same bandwidth service users and the same complementary acquisition time period can be used as a complementary acquisition task group, that is, the complementary acquisition tasks of the bandwidth service providers to be complemented and acquired in the same complementary acquisition time period by a bandwidth service user are divided into a complementary acquisition task group. Therefore, the complementary acquisition tasks belonging to the same bandwidth service user and having the same complementary acquisition time period can be intensively executed, so that the execution time difference is short, the bandwidth data of the bandwidth service user in the corresponding complementary acquisition time period can be checked again in time, and the checking efficiency of the bandwidth data of the bandwidth service user in the corresponding complementary acquisition time period is ensured.

A detailed description will be given below of a specific embodiment of determining a target node for executing all the complementary tasks in the complementary task group from the plurality of acquisition core nodes in S202. Specifically, the collection check nodes with more remaining resources may be preferentially allocated to execute all the complementary collection tasks in the complementary collection task group.

In one embodiment, for each acquisition check node, calculating the task weight of the acquisition check node according to the complementary acquisition time period and the complementary acquisition type of the complementary acquisition task to be executed in the complementary acquisition queue corresponding to the acquisition check node; then, determining the difference between the maximum weight corresponding to the collection check node and the task weight of the collection check node as the node weight of the collection check node, wherein the larger the node weight is, the larger the idle resource of the collection check node is; and finally, taking the collection check node with the larger node weight as a target node.

Specifically, the task weight of the collection check node may be determined by: aiming at each complementary acquisition task to be executed in the complementary acquisition queue corresponding to the acquisition core pair node, calculating a task sub-weight corresponding to the complementary acquisition task according to the complementary acquisition time period and the complementary acquisition type of the complementary acquisition task; and then, taking the sum of the task sub-weights corresponding to each to-be-executed complementary acquisition task in the complementary acquisition queue corresponding to the acquisition check node as the task weight of the acquisition check node.

For each to-be-executed complementary task, the task sub-weight = (the duration/preset time interval corresponding to the complementary period of the complementary task) corresponding to the complementary type of the complementary task, where the preset time interval is the bandwidth data acquisition time interval.

For example, for an acquisition check node L, the acquisition and compensation queue corresponding to the acquisition check node L has an acquisition and compensation task group 1 and an acquisition and compensation task group 2, wherein the acquisition and compensation task group 1 includes an acquisition and compensation task a to be executed and an acquisition and compensation task B to be executed, and the acquisition and compensation task group 2 includes an acquisition and compensation task C to be executed, wherein the acquisition and compensation periods of the acquisition and compensation task a, the acquisition and compensation task B, and the acquisition and compensation task C are respectively 0 point-1 point, and 3 point-4 points (i.e. the acquisition and compensation periods of the acquisition task a, the acquisition and compensation task B, and the acquisition and compensation task C are all 1 hour), the acquisition and compensation types of the acquisition and compensation task a, the acquisition and compensation tasks B, and the acquisition and compensation tasks C are respectively acquisition and compensation logs, acquisition and compensation off-line logs, and compensation off-line logs correspond to weights of 2 and acquisition bandwidth data of 1, the preset time interval was 5 minutes. In this way, task sub-weight = (1 h/5 min) × 2=24 for replenishment task a, task sub-weight = (1 h/5 min) × 1=12 for replenishment task B, and task sub-weight = (1 h/5 min) × 1=12 for replenishment task C. Therefore, the task weight of the acquisition and verification node L =48, which is the task sub-weight corresponding to the complementary task a + the task sub-weight corresponding to the complementary task B + the task sub-weight corresponding to the complementary task C.

In addition, because the time consumption for supplementing and acquiring the offline logs is long, the weight corresponding to the supplementing and acquiring offline logs is greater than the weight of the supplementing and acquiring bandwidth data, so that the node weight of the acquisition and checking node can be guaranteed to effectively measure the idle condition of the acquisition and checking node, and the execution efficiency of the supplementing and acquiring task is guaranteed.

Specifically, the acquisition and check node checks the bandwidth data through S301 to S303 shown in fig. 3.

In S301, in response to the real-time bandwidth data of the bandwidth interface of any bandwidth service provider acquired at the current acquisition time, the offline log of the bandwidth service provider and the historical bandwidth data of the bandwidth interface acquired at the previous acquisition time of the current acquisition time within the time period to which the current acquisition time belongs are acquired.

In the disclosure, after acquiring the real-time bandwidth data of the bandwidth interface of any bandwidth service provider acquired at the current acquisition time, the acquisition check node downloads the offline log of the bandwidth service provider within the time period to which the current acquisition time belongs from the log download interface provided by the bandwidth service provider, so that the offline log is analyzed to obtain the offline bandwidth data of the bandwidth interface at the current acquisition time.

In S302, a bandwidth data check is performed according to the offline log, the historical bandwidth data, and the real-time bandwidth data.

In S303, if the bandwidth data check fails, a complementary acquisition task for instructing to re-acquire the offline log of the bandwidth service provider or the real-time bandwidth data of the bandwidth interface at the current acquisition time in the time period to which the current acquisition time belongs is generated, and the complementary acquisition task is sent to the scheduling node, so that the scheduling node pushes the complementary acquisition task into a task queue.

For example, if the current collection time is 3 months, 25 days and 10:05, an offline log of the bandwidth service provider or a complementary collection task of the bandwidth interface for acquiring the real-time bandwidth data of the bandwidth interface at the 3 months, 25 days and 10:05 in the period of the 3 months, 25 days and 10:05 is generated.

In addition, when the bandwidth data check is failed, an abnormal alarm can be given.

A detailed description will be given below of a specific embodiment of performing the bandwidth data check based on the offline log, the historical bandwidth data, and the real-time bandwidth data in S302. Specifically, firstly, determining offline bandwidth data of the bandwidth interface at the current acquisition time according to an offline log, and specifically, acquiring the offline bandwidth data by analyzing the offline log; if the offline bandwidth data and the real-time bandwidth data meet a first preset condition, or the historical bandwidth data and the real-time bandwidth data meet a second preset condition, the bandwidth data are not checked; and if the offline bandwidth data and the real-time bandwidth data do not meet the first preset condition and the historical bandwidth data and the real-time bandwidth data do not meet the second preset condition, the bandwidth data are checked to pass.

In the present disclosure, the first preset condition may be any one of the following:

the absolute value of the difference between the offline bandwidth data and the real-time bandwidth data is greater than a first preset threshold;

the absolute value of the difference between the offline bandwidth data and the real-time bandwidth data/the offline bandwidth data is larger than a first preset proportion threshold;

the absolute value of the difference between the offline bandwidth data and the real-time bandwidth data is greater than a first preset threshold, and the absolute value of the difference between the offline bandwidth data and the real-time bandwidth data/the offline bandwidth data is greater than a first preset proportional threshold;

the absolute value of the difference between the offline bandwidth data and the real-time bandwidth data is greater than a first preset threshold, or the absolute value of the difference between the offline bandwidth data and the real-time bandwidth data/the offline bandwidth data is greater than a first preset proportion threshold.

The first preset threshold and the first preset proportion threshold may be determined according to the network bandwidth of the bandwidth interface, and the larger the network bandwidth is, the larger the first preset threshold and the first preset proportion threshold are, so that the accuracy of checking the bandwidth data may be ensured.

For example, the values of the first preset threshold and the first preset proportion threshold are as shown in table 1 below for different network bandwidths:

table 1 values of the first preset threshold and the first preset proportional threshold under different network bandwidths

Network bandwidth T	First preset threshold	First preset proportional threshold
			T≤100Mb/s	1Mb/s	1%
100Mb/s <T≤1Gb/s	10Mb/s	1.2%
			11Gb/s <T ≤10Gb/s	100Mb/s	1.2%
10Gb/s <T ≤100Gb/s	200Mb/s	1.2%
			100Gb/s <T ≤1Tb/s	500Mb/s	1.2%
T≥1Tb/s	1Gb/s	1.3%

The second preset condition may be any one of the following:

the absolute value of the difference between the historical bandwidth data and the real-time bandwidth data is greater than a second preset threshold;

the absolute value of the difference between the historical bandwidth data and the real-time bandwidth data/the historical bandwidth data is larger than a second preset proportion threshold;

the absolute value of the difference between the historical bandwidth data and the real-time bandwidth data is greater than a second preset threshold, and the absolute value of the difference between the historical bandwidth data and the real-time bandwidth data/the historical bandwidth data is greater than a second preset proportion threshold;

the absolute value of the difference between the historical bandwidth data and the real-time bandwidth data is larger than a second preset threshold, or the absolute value of the difference between the historical bandwidth data and the real-time bandwidth data/the historical bandwidth data is larger than a second preset proportion threshold.

The second preset threshold and the second preset proportion threshold may also be determined according to the network bandwidth of the bandwidth interface, and the larger the network bandwidth is, the larger the second preset threshold and the second preset proportion threshold are, so that the accuracy of checking the bandwidth data can be ensured.

Based on the specific embodiment of performing the bandwidth data check based on the offline log, the historical bandwidth data, and the real-time bandwidth data in S302, a specific embodiment of generating a complementary acquisition task for instructing to re-acquire the offline log of the bandwidth service provider or the real-time bandwidth data of the bandwidth interface at the current acquisition time in the time period to which the current acquisition time belongs if the bandwidth data check fails in S303 is described below.

And if the offline bandwidth data and the real-time bandwidth data meet the first preset condition and the offline bandwidth data are smaller than the real-time bandwidth data, generating a complementary collection task for indicating to collect the offline logs of the bandwidth service provider in the time period to which the current collection time belongs again.

And if any one of the following conditions is met, generating a complementary acquisition task for indicating to acquire the real-time bandwidth data of the bandwidth interface at the current acquisition time again: the historical bandwidth data and the real-time bandwidth data meet a second preset condition; the offline bandwidth data and the real-time bandwidth data meet a first preset condition, and the offline bandwidth data is larger than the real-time bandwidth data.

For example, if the historical bandwidth data and the real-time bandwidth data satisfy the second preset condition, the offline bandwidth data and the real-time bandwidth data satisfy the first preset condition, and the offline bandwidth data is smaller than the real-time bandwidth data, a complementary acquisition task for instructing to re-acquire the offline log of the bandwidth service provider in the time period to which the current acquisition time belongs is generated, and a complementary acquisition task for executing to re-acquire the real-time bandwidth data of the bandwidth interface at the current acquisition time is generated at the same time.

In addition, in order to further improve the accuracy of the bandwidth data check, when the bandwidth data check is performed, the comparable bandwidth data corresponding to the historical synchronization time of the bandwidth interface at the current acquisition time may be referred to. Specifically, the method may further include the steps of:

and determining the comparable bandwidth data corresponding to the historical contemporary moment of the bandwidth interface at the current acquisition moment.

In the present disclosure, the historical contemporaneous time may be the same acquisition time in the day before the current acquisition time, the same acquisition time in the previous week, and the like. Illustratively, the current acquisition time is 3 months and 25 days (thursday) 11:05, the historical synchronization time of the current acquisition time can be 3 months, 24 days, 11:05 minutes, or 11:05 days on last thursday (i.e., 3 months, 18 days).

In this case, in S302, the bandwidth data check may be performed according to the offline log, the historical bandwidth data, the real-time bandwidth data, and the parity bandwidth data.

A detailed description will be given below of a specific embodiment of the above-mentioned determination of the comparable bandwidth data corresponding to the historical synchronization time of the bandwidth interface at the current acquisition time.

In one embodiment, the bandwidth data of the bandwidth interface collected at the historical synchronization time can be directly used as the parity bandwidth data.

In another embodiment, the bandwidth data of the bandwidth interface, which is acquired at each historical acquisition time within the date of the historical synchronization time, may be acquired first; then, generating a fitting curve of the acquisition time and the bandwidth data according to the bandwidth data of the bandwidth interface acquired at each historical acquisition time; next, according to the fitting curve, determining fitting bandwidth data corresponding to the bandwidth interface at the historical synchronization time and fitting bandwidth data corresponding to a plurality of adjacent acquisition times (for example, the first N acquisition times and the last M acquisition times of the historical synchronization time) of the bandwidth interface at the historical synchronization time; and finally, determining the average value of all the fitting bandwidth data as the comparable bandwidth data corresponding to the bandwidth interface at the historical synchronous moment.

Illustratively, when the acquisition time is 3 months and 25 days (11: 05, the historical contemporaneous time of the current acquisition time may be 3 months and 24 days, 11:05 minutes (i.e., the same acquisition time of the previous day), the bandwidth data of the bandwidth interface, which is acquired at each historical acquisition time in the day before the current acquisition time, can be acquired firstly; then, the user can use the device to perform the operation, performing curve fitting according to the bandwidth data of the bandwidth interface, which is acquired at each historical acquisition time in the day before the current acquisition time, to obtain a fitting curve; next, the process of the present invention is described, according to a fitting curve, determining fitting bandwidth data corresponding to the bandwidth interface at the historical synchronization moment and the fitting bandwidth data corresponding to the bandwidth interface at each of the first 6 acquisition moments and the last 6 acquisition moments of the historical synchronization moment; finally, the process is carried out in a batch, and determining the average value of the 13 fitting bandwidth data as the comparable bandwidth data corresponding to the bandwidth interface at the historical synchronous moment.

The following describes in detail a specific embodiment of performing the bandwidth data check according to the offline log, the historical bandwidth data, the real-time bandwidth data, and the comparable bandwidth data. Specifically, firstly, determining offline bandwidth data of the bandwidth interface at the current acquisition time according to an offline log; determining that the bandwidth data check fails if any of the following conditions are met: the offline bandwidth data and the real-time bandwidth data meet a first preset condition, the historical bandwidth data and the real-time bandwidth data meet a second preset condition, and the comparable bandwidth data and the real-time bandwidth data meet a third preset condition; otherwise, determining that the bandwidth data check is passed.

In the present disclosure, the third preset condition may be any one of the following:

the absolute value of the difference between the comparable bandwidth data and the real-time bandwidth data is greater than a third preset threshold;

the absolute value of the difference between the bandwidth data with the same ratio and the real-time bandwidth data/the bandwidth data with the same ratio is larger than a third preset ratio threshold;

the absolute value of the difference between the bandwidth data with the same ratio and the real-time bandwidth data is greater than a third preset threshold value, and the absolute value of the difference between the bandwidth data with the same ratio and the real-time bandwidth data/the bandwidth data with the same ratio is greater than the third preset ratio threshold value;

the absolute value of the difference between the bandwidth data with the same ratio and the real-time bandwidth data is larger than a third preset threshold, or the absolute value of the difference between the bandwidth data with the same ratio and the real-time bandwidth data/the bandwidth data with the same ratio is larger than the third preset ratio threshold.

The third preset threshold and the third preset proportion threshold may also be determined according to the network bandwidth of the bandwidth interface, and the larger the network bandwidth is, the larger the third preset threshold and the third preset proportion threshold are, so that the accuracy of checking the bandwidth data can be ensured.

Based on the specific embodiment of performing the bandwidth data check according to the offline log, the historical bandwidth data, the real-time bandwidth data and the comparable bandwidth data in S302, a detailed description will be given of a specific embodiment of generating a complementary acquisition task for instructing to re-acquire the offline log of the bandwidth service provider or the real-time bandwidth data of the bandwidth interface at the current acquisition time in the time period to which the current acquisition time belongs if the bandwidth data check fails in S303.

And if any one of the following conditions is met, generating a complementary acquisition task for indicating to acquire the real-time bandwidth data of the bandwidth interface at the current acquisition time again:

the comparable bandwidth data and the real-time bandwidth data meet a third preset condition;

the offline bandwidth data and the real-time bandwidth data meet a first preset condition, and the offline bandwidth data is larger than the real-time bandwidth data.

Fig. 4 is a block diagram illustrating a bandwidth data collating apparatus according to an exemplary embodiment. The apparatus 400 is applied to a scheduling node in a bandwidth data checking system, where the bandwidth data checking system includes the scheduling node, a task queue node, and a plurality of collection checking nodes. As shown in fig. 4, the apparatus 400 includes: a grouping module 401, configured to obtain multiple complementary acquisition tasks popped up from a task queue, and group the multiple complementary acquisition tasks according to an association relationship between the multiple complementary acquisition tasks to obtain at least one complementary acquisition task group, where the complementary acquisition tasks are used to instruct to acquire offline logs or bandwidth data again, and when the acquisition check node determines that the bandwidth data check fails, a complementary acquisition task is generated and sent to the scheduling node, so that the scheduling node presses the complementary acquisition tasks into the task queue; a determining module 402, configured to determine, for each complementary acquisition task group obtained by the grouping module 401, a target node for executing all complementary acquisition tasks in the complementary acquisition task group from the multiple acquisition core nodes; a sending module 403, configured to send each complementary acquisition task group obtained by the grouping module 401 and the target node determined by the determining module 402 to execute all complementary acquisition tasks in each complementary acquisition task group to the task queue node.

the grouping module 401 is configured to use complementary tasks with the same bandwidth service users and the same complementary acquisition time period as a complementary acquisition task group.

Fig. 5 is a block diagram illustrating a bandwidth data collating apparatus according to an exemplary embodiment. The apparatus 500 is applied to any acquisition and check node in a bandwidth data check system, where the bandwidth data check system includes a scheduling node, a task queue node, and a plurality of the acquisition and check nodes. As shown in fig. 5, the apparatus 500 includes: an obtaining module 501, configured to respond to real-time bandwidth data of a bandwidth interface of any bandwidth service provider collected at a current collection time, and obtain an offline log of the bandwidth service provider and historical bandwidth data of the bandwidth interface collected at a previous collection time of the current collection time within a time period to which the current collection time belongs; a checking module 502, configured to perform bandwidth data checking according to the offline log, the historical bandwidth data, and the real-time bandwidth data acquired by the acquiring module 501; a generating module 503, configured to generate, if the checking module 502 determines that the bandwidth data is not checked, an additional acquisition task for instructing to re-acquire the offline log of the bandwidth service provider within the time period to which the current acquisition time belongs or the real-time bandwidth data of the bandwidth interface at the current acquisition time, and send the additional acquisition task to the scheduling node, so that the scheduling node presses the additional acquisition task into a task queue.

Optionally, the apparatus 500 further comprises: the bandwidth-sharing data determining module is used for determining bandwidth-sharing data corresponding to the historical synchronization moment of the bandwidth interface at the current acquisition moment; the checking module 502 is configured to perform bandwidth data checking according to the offline log, the historical bandwidth data, the real-time bandwidth data, and the comparable bandwidth data.

Optionally, the comparable bandwidth data determining module includes: the acquisition submodule is used for acquiring the bandwidth data of the bandwidth interface, which is acquired at each historical acquisition time in the date to which the historical synchronization time belongs; the first generation submodule is used for generating a fitting curve of the acquisition time and the bandwidth data according to the bandwidth data of the bandwidth interface acquired at each historical acquisition time; the first determining submodule is used for determining fitted bandwidth data corresponding to the bandwidth interface at the historical synchronization moment and fitted bandwidth data corresponding to the bandwidth interface at a plurality of adjacent acquisition moments of the historical synchronization moment according to the fitted curve; and the second determining submodule is used for determining the average value of all the fitting bandwidth data as the comparable bandwidth data corresponding to the bandwidth interface at the historical synchronization moment.

Optionally, the checking module 502 includes: a third determining submodule, configured to determine, according to the offline log, offline bandwidth data of the bandwidth interface at the current acquisition time; a fourth determining sub-module, configured to determine that the bandwidth data check fails if any one of the following conditions is satisfied: the offline bandwidth data and the real-time bandwidth data meet a first preset condition; the historical bandwidth data and the real-time bandwidth data meet a second preset condition; and the comparable bandwidth data and the real-time bandwidth data meet a third preset condition.

Optionally, the generating module 503 includes: a second generation submodule, configured to generate a complementary acquisition task for instructing to acquire an offline log of the bandwidth service provider again within a time period to which the current acquisition time belongs if the offline bandwidth data and the real-time bandwidth data meet the first preset condition and the offline bandwidth data are smaller than the real-time bandwidth data; a third generating sub-module, configured to generate a complementary acquisition task for instructing to reacquire real-time bandwidth data of the bandwidth interface at the current acquisition time if any one of the following conditions is met: the historical bandwidth data and the real-time bandwidth data meet the second preset condition; the geometric bandwidth data and the real-time bandwidth data meet the third preset condition; the offline bandwidth data and the real-time bandwidth data meet the first preset condition, and the offline bandwidth data is larger than the real-time bandwidth data.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

The present disclosure also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for checking bandwidth data on a scheduling node side or the steps of the method for checking bandwidth data on a collection checking node side provided by the present disclosure.

Fig. 6 is a block diagram illustrating an electronic device 600 according to an example embodiment. As shown in fig. 6, the electronic device 600 may include: a processor 601 and a memory 602. The electronic device 600 may also include one or more of a multimedia component 603, an input/output (I/O) interface 604, and a communications component 605.

The processor 601 is configured to control the overall operation of the electronic device 600, so as to complete all or part of the steps in the above-mentioned bandwidth data checking method on the scheduling node side. The memory 602 is used to store various types of data to support operation at the electronic device 600, such as instructions for any application or method operating on the electronic device 600 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and so forth. The Memory 602 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. The multimedia components 603 may include a screen and audio components. Where the screen may be, for example, a touch screen and the audio component is used to output or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 602 or transmitted through the communication component 605. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 604 provides an interface between the processor 601 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, which is not limited herein. The corresponding communication component 605 may therefore include: Wi-Fi module, Bluetooth module, NFC module, etc.

In an exemplary embodiment, the electronic Device 600 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, for performing the above-mentioned bandwidth data checking method on the scheduling node side.

In another exemplary embodiment, there is also provided a computer readable storage medium including program instructions, which when executed by a processor, implement the steps of the above-described bandwidth data checking method on the scheduling node side. For example, the computer readable storage medium may be the above-mentioned memory 602 including program instructions executable by the processor 601 of the electronic device 600 to perform the above-mentioned bandwidth data checking method on the scheduling node side.

Fig. 7 is a block diagram illustrating an electronic device 700 in accordance with an example embodiment. For example, the electronic device 700 may be provided as a server. Referring to fig. 7, an electronic device 700 includes a processor 722, which may be one or more in number, and a memory 732 for storing computer programs that are executable by the processor 722. The computer programs stored in memory 732 may include one or more modules that each correspond to a set of instructions. Further, the processor 722 may be configured to execute the computer program to perform the above-described bandwidth data checking method on the scheduling node side.

Additionally, the electronic device 700 may also include a power component 726 that may be configured to perform power management of the electronic device 700 and a communication component 750 that may be configured to enable communication, e.g., wired or wireless communication, of the electronic device 700. The electronic device 700 may also include input/output (I/O) interfaces 758. The electronic device 700 may operate based on an operating system, such as Windows Server, stored in the memory 732^TM，Mac OS X^TM，Unix^TM，Linux^TMAnd so on.

In another exemplary embodiment, there is also provided a computer readable storage medium including program instructions, which when executed by a processor, implement the steps of the above-described bandwidth data checking method on the scheduling node side. For example, the computer readable storage medium may be the memory 732 described above including program instructions executable by the processor 722 of the electronic device 700 to perform the bandwidth data checking method on the scheduling node side described above.

In another exemplary embodiment, a computer program product is also provided, which contains a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-mentioned bandwidth data checking method on the scheduling node side when executed by the programmable apparatus.

Fig. 8 is a block diagram illustrating an electronic device 800 in accordance with an example embodiment. As shown in fig. 8, the electronic device 800 may include: a processor 801, a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an input/output (I/O) interface 804, and a communications component 805.

The processor 801 is configured to control the overall operation of the electronic device 800, so as to complete all or part of the steps in the above method for acquiring and checking bandwidth data on a node side. The memory 802 is used to store various types of data to support operation at the electronic device 800, such as instructions for any application or method operating on the electronic device 800 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and so forth. The Memory 802 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. The multimedia components 803 may include screen and audio components. Where the screen may be, for example, a touch screen and the audio component is used to output or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 802 or transmitted through the communication component 805. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 804 provides an interface between the processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, which is not limited herein. The corresponding communication component 805 may therefore include: Wi-Fi module, Bluetooth module, NFC module, etc.

In an exemplary embodiment, the electronic Device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described bandwidth data checking method on the opposite side of the acquisition core.

In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described method of bandwidth data reconciliation on the contralateral side of acquisition reconciliation is also provided. For example, the computer readable storage medium may be the memory 802 described above that includes program instructions executable by the processor 801 of the electronic device 800 to perform the above described method of bandwidth data reconciliation on the contralateral side of the acquisition reconciliation.

The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, various possible combinations will not be separately described in this disclosure.

In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims

1. A method for checking bandwidth data is applied to a scheduling node in a bandwidth data checking system, wherein the bandwidth data checking system comprises the scheduling node, a task queue node and a plurality of acquisition checking nodes, and the method comprises the following steps:

2. The method of claim 1, wherein the complementary tasks include complementary periods and bandwidth service users;

3. A method for checking bandwidth data is applied to any acquisition checking node in a bandwidth data checking system, wherein the bandwidth data checking system comprises a scheduling node, a task queue node and a plurality of acquisition checking nodes, and is characterized by comprising the following steps:

4. The method of claim 3, further comprising:

5. The method of claim 4, wherein the determining comparable bandwidth data corresponding to the historical contemporaneous time of the bandwidth interface at the current acquisition time comprises:

6. The method of claim 4 or 5, wherein the performing a bandwidth data check based on the offline log, the historical bandwidth data, the real-time bandwidth data, and the parity bandwidth data comprises:

7. The method of claim 6, wherein if the bandwidth data check fails, generating a complementary task for instructing to re-collect the offline log of the bandwidth service provider or the bandwidth data of the bandwidth interface at the current collection time in the period of time to which the current collection time belongs comprises:

8. A bandwidth data check device is applied to a scheduling node in a bandwidth data check system, wherein the bandwidth data check system comprises the scheduling node, a task queue node and a plurality of acquisition check nodes, and the device comprises:

9. A bandwidth data check device is applied to any acquisition check node in a bandwidth data check system, wherein the bandwidth data check system comprises a scheduling node, a task queue node and a plurality of acquisition check nodes, and the device comprises:

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

11. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to carry out the steps of the method of claim 1 or 2.

12. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 3 to 7.