CN111447118B - Data transmission method and device based on data transmission stream - Google Patents
Data transmission method and device based on data transmission stream Download PDFInfo
- Publication number
- CN111447118B CN111447118B CN202010227180.2A CN202010227180A CN111447118B CN 111447118 B CN111447118 B CN 111447118B CN 202010227180 A CN202010227180 A CN 202010227180A CN 111447118 B CN111447118 B CN 111447118B
- Authority
- CN
- China
- Prior art keywords
- data
- file
- data transmission
- disk space
- size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/50—Overload detection or protection within a single switching element
- H04L49/501—Overload detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9084—Reactions to storage capacity overflow
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a data transmission method and a device based on a data transmission stream, wherein the data transmission method based on the data transmission stream comprises the following steps: acquiring a flow direction link of the data transmission flow; the flow direction link comprises at least one data sending node, at least one data transmission node and at least one data receiving node; setting at least one probe at the data sending node and the data transmission node; detecting the size of data to be received by the data receiving node according to the probe; and transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node. The invention can accurately early warn the use condition of the disk space in advance, the early warning accuracy is up to 100 percent, and the operation and maintenance labor cost and the financial cost consumed in the aspect of disk space management can be reduced practically.
Description
Technical Field
The invention relates to the technical field of information, in particular to the technical field of intelligent operation and maintenance and monitoring, and specifically relates to a data transmission method and device based on a data transmission stream.
Background
In the technical field of existing data transmission, the use condition of a traditional disk space depends on a threshold set by an operating system, and when the threshold is reached and an alarm is triggered, operation and maintenance personnel intervene in processing and are relatively passive. If the large data file received by the server exceeds the capacity of the disk, the whole file system cannot be written, so that the transaction is influenced, and once the situation occurs, the expansion of the disk space is not in time. In addition, the method of manually evaluating and expanding the disk space in advance is complex in combing, high labor cost is required, and the disk waste rate is high if general expansion is performed. With the development of internet finance, marketing activities launched on holidays and various promotion sections are increased day by day, which will cause the file generated by a certain transaction to be increased suddenly, and the difficulty of manual evaluation is increased more and more.
Disclosure of Invention
Aiming at the problems in the prior art, the data transmission method and the data transmission device based on the data transmission stream provided by the invention are used for identifying the size and the change rate of a file in advance at a source server side based on a data transmission path and sending the disk space information required by the file exceeding a configuration threshold value to a destination server in advance, thereby realizing the accurate early warning of the disk space use condition in advance. The early warning accuracy rate can reach 100% from the result of use, can cut down the fortune dimension manpower and financial resources cost that consume in the aspect of disk space management really.
In order to solve the technical problems, the invention provides the following technical scheme:
in a first aspect, the present invention provides a data transmission method based on a data transport stream, including:
acquiring a flow direction link of the data transmission flow; the flow direction link comprises at least one data sending node, at least one data transmission node and at least one data receiving node;
setting at least one probe at the data sending node and the data transmission node;
detecting the size of data to be received by the data receiving node according to the probe;
and transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node.
In one embodiment, the streaming link level is a file level.
In one embodiment, the data transmission method based on the data transmission stream further includes: initializing the flow direction link configuration information.
In an embodiment, the transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node includes:
judging the relation between the data size and the preset value of the disk space;
and transmitting the data to the data receiving node according to the judgment result.
In a second aspect, the present invention provides a data transmission apparatus based on data transmission stream, the apparatus comprising:
a stream link acquisition unit for acquiring a stream link of the data transport stream; the flow direction link comprises at least one data sending node, at least one data transmission node and at least one data receiving node;
a probe setting unit, configured to set at least one probe at the data sending node and the data transmission node;
the data detection unit is used for detecting the size of the data to be received by the data receiving node according to the probe;
and the data transmission unit is used for transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node.
In one embodiment, the streaming link level is a file level.
In one embodiment, the data transmission device based on data transmission stream further includes: a configuration information initialization unit, configured to initialize the flow direction link configuration information.
In one embodiment, the data transmission unit includes:
the relationship judgment module is used for judging the relationship between the data size and the preset value of the disk space;
and the data transmission module is used for transmitting the data to the data receiving node according to the judgment result.
In a third aspect, the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data transmission method based on the data transmission stream when executing the program.
In a fourth aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of a data transmission method based on a data transport stream.
As can be seen from the above description, the data transmission method and apparatus based on data transport stream according to the embodiments of the present invention obtain a file-level flow direction link based on a unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at a source server, so that a server where a whole transmission link node of the file is located can be detected in advance, and a required disk space can be determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a first flowchart illustrating a data transmission method based on data transmission streams according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a data transmission method based on data transport streams according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating steps 400 of a data transmission method based on data transmission streams according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating a data transmission method based on data transmission streams in an embodiment of the present invention;
FIG. 5 is a logic diagram of a data transmission method based on data transmission stream in an embodiment of the present invention;
FIG. 6 is a first schematic diagram of a data transmission device based on data transmission stream according to an embodiment of the present invention;
FIG. 7 is a second schematic structural diagram of a data transmission device based on data transmission streams in an embodiment of the present invention;
FIG. 8 is a diagram illustrating a structure of a data transmission unit according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device in an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In view of the rapid development of the internet financial industry, the traditional data transmission method cannot meet the data explosion growth caused by marketing activities launched in holidays and various promotion festivals, and based on this, the embodiment of the present invention provides a specific implementation manner of a data transmission method based on a data transmission stream, and referring to fig. 1, the method specifically includes the following contents:
step 100: acquiring a flow direction link of the data transmission flow; the streaming link includes at least one data transmitting node, and at least one data receiving node.
It should be understood that the data receiving point in step 100 may also be used as a data transmitting point, and correspondingly, the data transmitting point may also be used as a data receiving point, which is not limited in this embodiment.
Step 200: and arranging at least one probe at the data sending node and the data transmission node.
It can be understood that the probe is used for acquiring information required by work from massive data, and further providing an information early warning prompt, specifically: the probe is special equipment for acquiring data from a router and a switch. The probe is divided into an internal probe and an external probe, and the internal probe means that probe equipment and existing telecommunication equipment are in the same machine frame and directly acquire data. The external probe means that most network equipment in the existing network is already deployed and cannot move the original network, and the external probe is needed at this time. The probe (network probe) can detect other computers in a specific network interval (generally, the same C-type network segment of an internal network) and take technical measures to automatically limit, detect or block the network communication of the computers. These operations are performed automatically by the network probe without manual operations by the administrator and without modifications to the configuration of the associated network equipment.
Step 300: and detecting the size of the data to be received by the data receiving node according to the probe.
Step 400: and transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node.
It can be understood that, for the data receiving node, it is important to advance the size of the file to be received for the disk space control itself.
As can be seen from the above description, the data transmission method based on the data transport stream according to the embodiments of the present invention obtains the file-level flow direction link based on the unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at the source server, so that the server where the whole transmission link node of the file is located can be detected in advance, and the required disk space can be determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
In one embodiment, the streaming link level is a file level.
It is understood that the file level refers to a file system, a single file may be composed of one or more logical blocks, the logical blocks are discontinuously distributed, the logical blocks are greater than or equal to an integral multiple of the physical blocks, and in contrast, the block level refers to a sector basis, and one or more continuous sectors are distributed between the file system and a block device (disk drive), which is also called a physical block.
In an embodiment, referring to fig. 2, the data transmission method based on the data transmission stream further includes:
step 500: initializing the flow direction link configuration information.
When the step 500 is implemented, it is specifically: and initializing data flow configuration information according to file call requirements registered by each demand party. And configuring the data flow direction of the related files by using the unified data exchange platform according to the calling requirements of each platform application on the files, wherein the data flow direction specifically comprises the file name, the server information for receiving the files, the disk file system for receiving the files and other basic information.
In one embodiment, referring to fig. 3, step 400 further comprises:
step 401: and judging the relation between the data size and the preset value of the disk space.
Specifically, threshold configuration can be performed on the size of the file received by each receiver and the change rate of the file size, and the two threshold items are in an OR relationship, so that the two threshold items are used as an early warning rule for judgment. For example, for a data disk with an IP of 192.168.1.1, when the size of the received file is 1GB and the change rate of the size of the received file is 50%, it means that when the size of the received file of the data disk is greater than or equal to 1GB or the change rate of the size of the received file is greater than or equal to 50%, the relevant action of the disk space detection module is automatically triggered.
Step 402: and transmitting the data to the data receiving node according to the judgment result.
It can be understood that when the data size is within the range of the disk space threshold of the data receiving node, the data is sent to the data receiving node, whereas when the data size is outside the range of the threshold of the data receiving node, the early warning information is sent to the data sending node.
As can be seen from the above description, the data transmission method based on the data transport stream according to the embodiments of the present invention obtains the file-level flow direction link based on the unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at the source server, so that the server where the whole transmission link node of the file is located can be detected in advance, and the required disk space can be determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
To further illustrate the present solution, the present invention provides a specific application example of the data transmission method based on the data transmission stream by taking a unified data exchange platform in the banking system as an example, and the specific application example specifically includes the following contents, see fig. 4 and fig. 5.
S0: the data flow direction is configured.
And initializing data flow configuration information according to file call requirements registered by each demand party. Specifically, a unified data exchange platform of a bank is utilized, and according to the calling requirements of each platform application on files, the data flow direction of related files is configured, wherein the data flow direction specifically comprises the file name, the server information for receiving the files, the disk file system for receiving the files and other basic information.
S1: and configuring an early warning rule.
And configuring an early warning threshold value in the unified data exchange platform by operation and maintenance personnel of each receiver. Preferably, a foreground with an early warning threshold value configuration function is provided, and operation and maintenance personnel can perform threshold value configuration on the size of the received file and the change rate of the size of the file of each receiver, wherein the two threshold value items are in an OR relationship and serve as early warning rules to be judged by a following module. For example, for a data disk with an IP of 192.168.1.1, when the size of the received file is 1GB and the change rate of the size of the received file is 50%, it means that when the size of the received file of the data disk is greater than or equal to 1GB or the change rate of the size of the received file is greater than or equal to 50%, the related action of disk space detection is automatically triggered.
S2: and acquiring server information for receiving the file and a disk file system for receiving the file from the data flow configuration information according to the file name, and acquiring an early warning threshold value from early warning rule configuration according to the basic information.
S3: and judging whether the early warning threshold value is met or not according to the size of the file.
It can be understood that if the early warning threshold is met, disk space detection is initiated to a receiver; and if the file is small and the change rate is small and does not meet the early warning threshold value, executing the step 8 to transmit data.
If the size of the data file meets the threshold condition, the agent deployed on the server receiving the file is called, and the server receiving the file is detected according to the information configured in the step S0, wherein the detection initiating information mainly includes the total capacity and the used capacity of the disk space receiving the file. The receiving party obtains the utilization rate of the disk space after receiving the file according to the following calculation rules:
according to the calculation result, the receiving party judges whether the file can be received. If the utilization rate of the disk space after receiving the file is less than 85% (the utilization rate can be adjusted according to actual conditions, but a certain margin needs to be reserved), returning a detection result true to the sender, and indicating that the disk space of the receiver is satisfied; otherwise, returning the detection result false to the sender, which indicates that the disk space of the receiver is not enough.
S4: and carrying out data transmission according to the result returned by the disk space detection.
Specifically, according to the returned result of the disk space detection, if the result is false, a corresponding monitoring alarm is sent for operation and maintenance personnel to intervene in the processing in time. The sender executes the step 4 again after 10 minutes; if true (until the disk space of the receiving party meets the condition), the sending party transmits the file to the downstream by using the file transmission agent.
As can be seen from the above description, the data transmission method based on the data transport stream according to the embodiments of the present invention obtains the file-level flow direction link based on the unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at the source server, so that the server where the whole transmission link node of the file is located can be detected in advance, and the required disk space can be determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
Based on the same inventive concept, the embodiments of the present application further provide a data transmission apparatus based on a data transport stream, which can be used to implement the methods described in the above embodiments, such as the following embodiments. Because the principle of solving the problem of the data transmission device based on the data transport stream is similar to that of the data transmission method based on the data transport stream, the implementation of the data transmission device based on the data transport stream can refer to the implementation of the data transmission method based on the data transport stream, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
An embodiment of the present invention provides a specific implementation of a data transmission device based on a data transport stream, which is capable of implementing a data transmission method based on a data transport stream, and referring to fig. 6, the data transmission device based on a data transport stream specifically includes the following contents:
a stream link acquiring unit 10, configured to acquire a stream link of the data transport stream; the streaming link includes at least one data transmitting node, and at least one data receiving node.
A probe setting unit 20, configured to set at least one probe at the data sending node and the data transmission node.
A data detecting unit 30, configured to detect the size of the data to be received by the data receiving node according to the probe.
And a data transmission unit 40, configured to transmit the data to the data receiving node according to the data size and the disk space of the data receiving node.
In one embodiment, the streaming link level is a file level.
In one embodiment, referring to fig. 7, the data transmission device based on data transmission stream further includes: a configuration information initializing unit 50, configured to initialize the flow direction link configuration information.
In one embodiment, referring to fig. 8, the data transmission unit 40 includes:
a relationship determining module 401, configured to determine a relationship between the data size and the preset value of the disk space.
A data transmission module 402, configured to transmit the data to the data receiving node according to the determination result.
As can be seen from the above description, the data transmission device based on the data transport stream according to the embodiment of the present invention obtains the file-level flow direction link based on the unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at the source server, so that the server where the whole transmission link node of the file is located can be detected in advance, and the required disk space can be determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
An embodiment of the present application further provides a specific implementation manner of an electronic device, which is capable of implementing all steps in the data transmission method based on the data transport stream in the foregoing embodiment, and referring to fig. 9, the electronic device specifically includes the following contents:
a processor (processor)1201, a memory (memory)1202, a communication Interface 1203, and a bus 1204;
the processor 1201, the memory 1202 and the communication interface 1203 complete communication with each other through the bus 1204; the communication interface 1203 is used for implementing information transmission between related devices such as a server-side device, a probe, a client-side device, and the like.
The processor 1201 is configured to call the computer program in the memory 1202, and the processor executes the computer program to implement all the steps in the data transmission stream-based data transmission method in the above embodiments, for example, the processor executes the computer program to implement the following steps:
step 100: acquiring a flow direction link of the data transmission flow; the streaming link includes at least one data transmitting node, and at least one data receiving node.
Step 200: and arranging at least one probe at the data sending node and the data transmission node.
Step 300: and detecting the size of the data to be received by the data receiving node according to the probe.
Step 400: and transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node.
As can be seen from the above description, the electronic device in the embodiment of the present application obtains the file-level flow direction link based on the unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at the source server, so that the server where the whole transmission link node of the file is located can be detected in advance, and the required disk space is determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
Embodiments of the present application further provide a computer-readable storage medium capable of implementing all steps in the data transmission stream-based data transmission method in the foregoing embodiments, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements all steps of the data transmission stream-based data transmission method in the foregoing embodiments, for example, when the processor executes the computer program, the processor implements the following steps:
step 100: acquiring a flow direction link of the data transmission flow; the streaming link includes at least one data transmitting node, and at least one data receiving node.
Step 200: and arranging at least one probe at the data sending node and the data transmission node.
Step 300: and detecting the size of the data to be received by the data receiving node according to the probe.
Step 400: and transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node.
As can be seen from the above description, the computer-readable storage medium in the embodiment of the present application obtains a file-level flow link based on a unified data exchange platform, that is, when a file is generated, the size and the change rate of the file are identified in advance at a source server, so that a server where a whole transmission link node of the file is located can be detected in advance, and a required disk space is determined. When the size or the change rate of the file reaches a certain degree, the detection method is triggered, and the disk space information required by the file exceeding the configuration threshold value is sent to the destination server in advance, so that the accurate early warning of the disk space use condition is realized in advance. The method can accurately achieve the data size to the file level, achieve the purpose of predicting the disk space required by the downstream server in advance, reserve disk capacity expansion time for the downstream server, and effectively prevent the transaction damage caused by insufficient disk space. By utilizing the accuracy and the early warning performance of the invention, the disk waste caused by the universal capacity expansion can be avoided, the early warning accuracy is up to 100 percent from the view of using effect, and the operation and maintenance manpower and financial cost consumed in the aspect of disk space management can be reduced practically.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Although the present application provides method steps as in an embodiment or a flowchart, more or fewer steps may be included based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or client product executes, it may execute sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (7)
1. A data transmission method based on a data transmission stream, comprising:
acquiring a flow direction link of the data transmission flow; the flow direction link comprises at least one data sending node, at least one data transmission node and at least one data receiving node;
setting at least one probe at the data sending node and the data transmission node;
detecting the size of data to be received by the data receiving node according to the probe;
transmitting the data to the data receiving node according to the size of the data and the disk space of the data receiving node;
the flow direction link level is a file level, the file level refers to a file system, a single file consists of one or more logic blocks, the logic blocks are distributed discontinuously, and the logic blocks are greater than or equal to integral multiples of physical blocks;
the moment when the probe detects the size of the data is the time when the data is generated;
the transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node includes:
judging the relation between the data size and the preset value of the disk space;
transmitting the data to the data receiving node according to the judgment result;
the judging the relationship between the data size and the preset value of the disk space and transmitting the data to the data receiving node according to the judging result comprises the following steps:
if the file size meets the early warning threshold, initiating disk space detection to a receiver; if the file is small and the change rate of the file size is small and does not meet the early warning threshold value, data transmission is carried out;
if the size of the data file meets the early warning threshold value, calling an agent deployed on a server for receiving the file, and initiating detection on the server for receiving the file according to the data flow direction configuration information, wherein the detection initiation information is the total capacity and the used capacity of the disk space for receiving the file; the receiving party obtains the utilization rate of the disk space after receiving the file according to the following calculation rules:
according to the calculation result, the receiver judges whether the file can be received, if the utilization rate of the disk space after the file is received is less than 85%, the detection result true is returned to the sender, and the requirement of the disk space of the receiver is indicated; otherwise, returning the detection result false to the sender, which indicates that the disk space of the receiver is not enough.
2. The data transmission method based on the data transmission stream as claimed in claim 1, further comprising: initializing the flow direction link configuration information.
3. A data transmission apparatus based on a data transmission stream, comprising:
a stream link acquisition unit for acquiring a stream link of the data transport stream; the flow direction link comprises at least one data sending node, at least one data transmission node and at least one data receiving node;
a probe setting unit, configured to set at least one probe at the data sending node and the data transmission node;
the data detection unit is used for detecting the size of the data to be received by the data receiving node according to the probe;
the data transmission unit is used for transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node;
the flow direction link level is a file level, the file level refers to a file system, a single file consists of one or more logic blocks, the logic blocks are distributed discontinuously, and the logic blocks are greater than or equal to integral multiples of physical blocks;
the moment when the probe detects the size of the data is the time when the data is generated;
the transmitting the data to the data receiving node according to the data size and the disk space of the data receiving node includes:
judging the relation between the data size and the preset value of the disk space;
transmitting the data to the data receiving node according to the judgment result;
the judging the relationship between the data size and the preset value of the disk space and transmitting the data to the data receiving node according to the judging result comprises the following steps:
if the file size meets the early warning threshold, initiating disk space detection to a receiver; if the file is small and the change rate of the file size is small and does not meet the early warning threshold value, data transmission is carried out;
if the size of the data file meets the early warning threshold value, calling an agent deployed on a server for receiving the file, and initiating detection on the server for receiving the file according to the data flow direction configuration information, wherein the detection initiation information is the total capacity and the used capacity of the disk space for receiving the file; the receiving party obtains the utilization rate of the disk space after receiving the file according to the following calculation rules:
according to the calculation result, the receiver judges whether the file can be received, if the utilization rate of the disk space after the file is received is less than 85%, the detection result true is returned to the sender, and the requirement of the disk space of the receiver is indicated; otherwise, returning the detection result false to the sender, which indicates that the disk space of the receiver is not enough.
4. The data transmission device according to claim 3, further comprising: a configuration information initialization unit, configured to initialize the flow direction link configuration information.
5. The data transmission device according to claim 3, wherein the data transmission unit comprises:
the relationship judgment module is used for judging the relationship between the data size and the preset value of the disk space;
and the data transmission module is used for transmitting the data to the data receiving node according to the judgment result.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the data transmission stream based data transmission method according to any one of claims 1 to 2 when executing the program.
7. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the data transmission method based on a data transport stream according to any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010227180.2A CN111447118B (en) | 2020-03-27 | 2020-03-27 | Data transmission method and device based on data transmission stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010227180.2A CN111447118B (en) | 2020-03-27 | 2020-03-27 | Data transmission method and device based on data transmission stream |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111447118A CN111447118A (en) | 2020-07-24 |
| CN111447118B true CN111447118B (en) | 2022-04-15 |
Family
ID=71650864
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010227180.2A Active CN111447118B (en) | 2020-03-27 | 2020-03-27 | Data transmission method and device based on data transmission stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111447118B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001069476A (en) * | 1999-08-26 | 2001-03-16 | Casio Comput Co Ltd | Network connection system provided with charging function |
| CN101873299A (en) * | 2009-04-24 | 2010-10-27 | 北京兴大豪科技开发有限公司 | Serial bus, communication method and system |
| CN102025515A (en) * | 2009-09-14 | 2011-04-20 | 阿里巴巴集团控股有限公司 | File directory based file transmission method, device and system |
| CN102412875A (en) * | 2011-12-26 | 2012-04-11 | 中兴通讯股份有限公司 | File sending and receiving method and device as well as file transmission method and system |
| CN109660973A (en) * | 2018-12-28 | 2019-04-19 | 努比亚技术有限公司 | Bluetooth control method, mobile terminal and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007323428A (en) * | 2006-06-01 | 2007-12-13 | Hitachi Ltd | Bot detection apparatus, bot detection method and program |
| CN105933150A (en) * | 2016-04-20 | 2016-09-07 | 努比亚技术有限公司 | OTA upgrade method, device and system |
-
2020
- 2020-03-27 CN CN202010227180.2A patent/CN111447118B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001069476A (en) * | 1999-08-26 | 2001-03-16 | Casio Comput Co Ltd | Network connection system provided with charging function |
| CN101873299A (en) * | 2009-04-24 | 2010-10-27 | 北京兴大豪科技开发有限公司 | Serial bus, communication method and system |
| CN102025515A (en) * | 2009-09-14 | 2011-04-20 | 阿里巴巴集团控股有限公司 | File directory based file transmission method, device and system |
| CN102412875A (en) * | 2011-12-26 | 2012-04-11 | 中兴通讯股份有限公司 | File sending and receiving method and device as well as file transmission method and system |
| CN109660973A (en) * | 2018-12-28 | 2019-04-19 | 努比亚技术有限公司 | Bluetooth control method, mobile terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111447118A (en) | 2020-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102134547B1 (en) | Risk identification method, risk identification device, and cloud risk identification device and system | |
| US11818014B2 (en) | Multi-baseline unsupervised security-incident and network behavioral anomaly detection in cloud-based compute environments | |
| CN106682906B (en) | Risk identification and service processing method and equipment | |
| US20200204576A1 (en) | Automated determination of relative asset importance in an enterprise system | |
| CN111711623A (en) | Data verification method and device | |
| CN112637162A (en) | UDP (user Datagram protocol) data packet processing method and device | |
| CN114363212B (en) | Equipment detection method, device, equipment and storage medium | |
| US11410049B2 (en) | Cognitive methods and systems for responding to computing system incidents | |
| CN113641544B (en) | Method, apparatus, device, medium and product for detecting application state | |
| US10540594B2 (en) | Identifying abnormal pumpjack conditions | |
| CN111159009B (en) | Pressure testing method and device for log service system | |
| CN113495825A (en) | Line alarm processing method and device, electronic equipment and readable storage medium | |
| CN111447118B (en) | Data transmission method and device based on data transmission stream | |
| CN112235300A (en) | Cloud virtual network vulnerability detection method, system and device and electronic equipment | |
| CN110442439B (en) | Task process processing method and device and computer equipment | |
| CN117170969A (en) | Operation and maintenance method, partition equilibrium recovery device, equipment and medium of server cluster | |
| CN112433908A (en) | Method, system, device and medium for determining interval time of detection server | |
| CN117093627A (en) | Information mining method, device, electronic equipment and storage medium | |
| CN116595523A (en) | Multi-engine file detection method, system, equipment and medium based on dynamic arrangement | |
| CN116361153A (en) | Method and device for testing firmware codes, electronic equipment and storage medium | |
| CN114116128B (en) | Container instance fault diagnosis method, device, equipment and storage medium | |
| CN113330716B (en) | Method and device for determining dependency relationship of application service and processor | |
| CN115269519A (en) | Log detection method and device and electronic equipment | |
| CN107025148A (en) | A kind for the treatment of method and apparatus of mass data | |
| CN115600195A (en) | Web attack detection method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |