CN113542424B - Data processing method, device, equipment and readable storage medium - Google Patents
Data processing method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN113542424B CN113542424B CN202110816540.7A CN202110816540A CN113542424B CN 113542424 B CN113542424 B CN 113542424B CN 202110816540 A CN202110816540 A CN 202110816540A CN 113542424 B CN113542424 B CN 113542424B
- Authority
- CN
- China
- Prior art keywords
- salt
- client
- minion
- preset
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a data processing method, which comprises the following steps: the method comprises the steps that a salt-master server side obtains system information sent by each salt-minion client side, wherein a monitoring analysis unit of the salt-minion client side obtains the system information at regular time, and feeds back the system information to the salt-master server side; the monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition or not based on the system information; and if the fact that the target client which reaches the preset alarm condition exists in the salt-min client is determined, the monitoring analysis module outputs alarm prompt information based on the target client. The invention also discloses a data processing device, equipment and a computer program product. According to the invention, the monitoring analysis module of the salt-master server side is used for timely analyzing the states of the salt-minion clients, so that maintenance personnel can be timely notified when the salt-minion clients are down or the network is disconnected, and the reliability of the system is improved.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a data processing method, apparatus, device, and computer program product.
Background
The CDN (Content Delivery Network ) delivers the content in the origin server to the distributed deployed service nodes, and by applying the traffic scheduling technique, the user request can be automatically diverted to the globally optimal service node according to a specified policy, and the node provides service for the user. In the service process, operations such as cache refreshing (deleting the cached out-of-date content), cache preheating (caching the content of the source station in advance and reducing the pressure of the source station when the user accesses concurrently) and the like are needed to be performed on the cache content in the edge node of the actual service.
In a content distribution network, thousands of servers are often deployed as edge nodes, and accurate management of the servers and scheduling of each node for content management (including cache refreshing, cache preheating, etc.) are required. At present, task servers are usually scheduled uniformly through a deployment center so as to achieve content management operation, however, the state of a machine cannot be monitored in the mode, so that operation and maintenance personnel cannot timely sense when the servers are down or a network is disconnected, and the system reliability is low.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a data processing method, a device, equipment and a computer program product, which aim to solve the problem that the reliability of a system is low because the existing content distribution network cannot monitor the state of a server in time.
To achieve the above object, the present invention provides a data processing method including the steps of:
the method comprises the steps that a salt-master server side obtains system information sent by each salt-master client side, wherein the system information comprises CPU (central processing unit) utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, a monitoring analysis unit of the salt-master client side obtains the system information at regular time, and feeds back the system information to the salt-master server side;
the monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition or not based on the system information;
and if the fact that the target client which reaches the preset alarm condition exists in the salt-min client is determined, the monitoring analysis module outputs alarm prompt information based on the target client.
Further, the step of determining whether the salt-min client reaches a preset alarm condition by the monitoring analysis module of the salt-master server based on the system information includes:
the monitoring analysis module of the salt-master server determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth;
and if the CPU utilization rate reaches the preset CPU utilization rate, the memory utilization rate reaches the preset memory utilization rate, the disk utilization amount reaches the preset disk utilization amount or the network card bandwidth reaches the preset bandwidth, the monitoring analysis module of the salt-master server determines that the salt-minion client reaches the preset alarm condition.
Further, the data processing method further comprises:
if the system information sent by the first salt-min client is not received within the first preset time, the salt-master server sets the state of the first salt-min client as an unavailable state.
Further, the step of determining whether the salt-min client reaches a preset alarm condition by the monitoring analysis module of the salt-master server based on the system information includes:
if a second salt-minion client in an unavailable state exists in the salt-minion clients, the monitoring analysis module sets the state of the second salt-minion client to be an on-line state;
and the monitoring analysis module determines whether the salt-min client reaches a preset alarm condition or not based on the system information.
Further, the data processing method further comprises:
the method comprises the steps that a content management module of a salt-master server receives a task request sent by a management platform of the salt-master server, wherein the task request comprises a content management type and a resource address;
the content management module determines a target salt-minion client corresponding to the task request from each salt-minion client based on the state information of the salt-minion client;
the content management module sends a target task to the target salt-minion client based on the content management type and the resource address, wherein when the target task is received, the target salt-minion client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
and the management platform displays the operation result.
Further, the step of the content management module sending the target task to the target salt-minimum client based on the content management type and the resource address further includes:
if the running result fed back by the target salt-minion client is not received within a second preset time period, the content management module updates retry times;
if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send a target task to the target salt-minimum client based on the content management type and the resource address;
and if the retry times reach the preset retry times, the content management module outputs alarm information.
Further, after the step of outputting the alarm prompt information by the monitoring analysis module if the salt-min client reaches the preset alarm condition, the method further includes:
the monitoring analysis module acquires state information of the salt-minion client, wherein the state information comprises an online state and an unavailable state;
the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server;
and the management platform displays the system information and the state information of each salt-minion client.
In addition, in order to achieve the above object, the present invention also provides a data processing device, applied to a salt-master server, the data processing device comprising:
the system comprises an acquisition module, a monitoring analysis unit and a feedback module, wherein the acquisition module is used for acquiring system information sent by each salt-minion client, wherein the system information comprises CPU (central processing unit) utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, and a monitoring analysis unit of the salt-minion client acquires the system information at regular time and feeds back the system information;
the monitoring analysis module is used for determining whether the salt-min client reaches a preset alarm condition or not based on the system information;
and the output module is used for outputting alarm prompt information based on the target client if the target client reaching the preset alarm condition exists in the salt-minimum client.
In addition, in order to achieve the above object, the present invention also provides a data processing apparatus including: the system comprises a memory, a processor and a data processing program stored in the memory and capable of running on the processor, wherein the data processing program realizes the steps of the data processing method when being executed by the processor.
Furthermore, to achieve the above object, the present invention provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the aforementioned data processing method.
The method comprises the steps that system information sent by each salt-min client is obtained through a salt-master server, wherein the system information comprises CPU utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, and a monitoring analysis unit of the salt-min client regularly obtains the system information and feeds back the system information to the salt-master server; then, a monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition or not based on the system information; and if the target client which reaches the preset alarm condition exists in the salt-min client, the monitoring analysis module outputs alarm prompt information based on the target client, and the state of each salt-min client is analyzed in time through the monitoring analysis module of the salt-master server, so that maintenance personnel can be notified in time when the salt-min client is down or disconnected from the network, and the reliability of the system is improved.
Drawings
FIG. 1 is a schematic diagram of a data processing apparatus of a hardware operating environment in which embodiments of the present invention are directed;
FIG. 2 is a flow chart of a first embodiment of a data processing method according to the present invention;
FIG. 3 is a functional block diagram of a data processing apparatus according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
With reference to FIG. 1, FIG. 1 is a schematic diagram of a data processing apparatus of a hardware runtime environment according to an embodiment of the present invention.
The data processing device of the embodiment of the invention can be a PC, or can be a mobile terminal device with a display function, such as a smart phone, a tablet personal computer, an electronic book reader, an MP3 (Moving PictureExperts Group Audio Layer III, dynamic image expert compression standard audio layer 3) player, an MP4 (Moving PictureExperts Group Audio Layer IV, dynamic image expert compression standard audio layer 4) player, a portable computer and the like.
As shown in fig. 1, the data processing apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Optionally, the data processing device may also include a camera, RF (radio frequency) circuitry, sensors, audio circuitry, wiFi modules, and the like. Among other sensors, such as light sensors, motion sensors, and other sensors. Of course, the data processing device may also be configured with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, and the like, which are not described herein.
It will be appreciated by those skilled in the art that the terminal structure shown in fig. 1 does not constitute a limitation of the data processing apparatus and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a data processing program may be included in the memory 1005, which is a type of computer storage medium.
In the data processing apparatus shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server, and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be used to invoke data processing programs stored in the memory 1005.
In this embodiment, a data processing apparatus includes: the data processing system comprises a memory 1005, a processor 1001 and a data processing program stored in the memory 1005 and capable of running on the processor 1001, wherein the processor 1001 executes the steps of the data processing method in the following embodiments when calling the data processing program stored in the memory 1005.
The present invention also provides a data processing method, referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data processing method of the present invention.
In this embodiment, the data processing method includes the following steps:
step S101, a salt-master server acquires system information sent by each salt-master client, wherein the system information comprises CPU (Central processing Unit) utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, a monitoring analysis unit of the salt-master client acquires the system information at regular time, and feeds back the system information to the salt-master server;
in this embodiment, saltStack is applied to the CDN, and is a server infrastructure centralized management platform, which is a configuration management system that can maintain remote nodes in defined states (e.g., ensure that specific software packages are installed and specific services are run). SaltStack is also a distributed remote execution system for executing commands and querying data individually or by arbitrary selection on remote nodes. The SaltStack is divided into a server and a client, all managed servers are installed with Salt-minion and run, in addition, one or more hosts are used as the server to install Salt-master, and instructions can be sent to the Salt-minion through command lines. The Salt-minion establishes a TCP connection with the Salt-master through the message queue component zeroMQ. The Salt-master can publish tasks through a publish/subscribe mode of the message queue, and the Salt-minion can send information and feed back execution results to the Salt-master through a request/corresponding mode of the message queue. That is, the salt-master server is a host (main server) in the CDN, and each salt-minion client is a managed server in the CDN.
In this embodiment, each salt-min client is provided with a monitoring analysis unit, and the monitoring analysis unit periodically acquires the system information, where the system information includes a CPU usage rate, a memory usage rate, a disk usage amount, and a network card bandwidth, that is, each monitoring analysis unit periodically acquires the CPU usage rate, the memory usage rate, the disk usage amount, and the network card bandwidth of the salt-min client to which it belongs, and feeds back the system information to the salt-master server. And the salt-master server acquires system information sent by each salt-minion client.
The time interval for acquiring the system information regularly can be set reasonably so as to realize the timing monitoring of the salt-minion client, for example, the time interval can be 1 minute and the like.
Step S102, a monitoring analysis module of a salt-master server determines whether the salt-min client reaches a preset alarm condition or not based on the system information;
in this embodiment, the salt-master server is provided with a monitoring analysis module, and the monitoring analysis module is configured to analyze the received system information of each salt-minion client, that is, the monitoring analysis module determines, based on the system information, whether the salt-minion client reaches a preset alarm condition, so as to monitor the state of the machine.
Step S103, if it is determined that a target client reaching a preset alarm condition exists in the salt-min client, the monitoring analysis module outputs alarm prompt information based on the target client.
In this embodiment, whether the preset alarm condition is reached in each salt-min client is determined through the system information, if it is determined that a target client reaching the preset alarm condition exists in the salt-min client, the monitoring analysis module obtains client information corresponding to the target client and outputs alarm prompt information based on the client information, specifically, a preset mailbox or a preset mobile terminal corresponding to an operation and maintenance person of the target client can be obtained in a management platform of the salt-master server, and alarm prompt information is output to the preset mailbox or the preset mobile terminal based on the client information, so that the operation and maintenance person can process the target client in time, and management of each client in the CDN system is facilitated.
Further, in one embodiment, the step S102 includes:
the monitoring analysis module of the salt-master server determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth;
and if the CPU utilization rate reaches the preset CPU utilization rate, the memory utilization rate reaches the preset memory utilization rate, the disk utilization amount reaches the preset disk utilization amount or the network card bandwidth reaches the preset bandwidth, the monitoring analysis module of the salt-master server determines that the salt-minion client reaches the preset alarm condition.
In this embodiment, when system information of the salt-minion client is obtained, the monitoring analysis module determines whether the CPU usage reaches a preset CPU usage, whether the memory usage reaches a preset memory usage, whether the disk usage reaches a preset disk usage or whether the network card bandwidth reaches a preset bandwidth, respectively, if the CPU usage reaches the preset CPU usage, the memory usage reaches the preset memory usage, the disk usage reaches the preset disk usage or the network card bandwidth reaches the preset bandwidth, that is, if any parameter of the CPU usage, the memory usage, the disk usage and the network card bandwidth reaches a corresponding preset condition, the monitoring analysis module determines that the salt-minion client reaches a preset alarm condition, and further can accurately determine whether the salt-minion client reaches the preset alarm condition according to the system information, so as to realize accurate monitoring of each salt-minion client.
The preset CPU utilization, the preset memory utilization, the preset disk utilization, the preset bandwidth, and the like may be reasonably set, for example, the preset CPU utilization is 80% and the preset memory utilization is 80%.
Further, in an embodiment, the data processing method further includes:
step a, if the system information sent by the first salt-min client is not received within a first preset time period, the salt-master server sets the state of the first salt-min client as an unavailable state.
In this embodiment, for each salt-min client, when receiving system information, the salt-master server sets a timer of a first preset duration to determine whether system information of the salt-min client is received within the first preset duration, if system information sent by the first salt-min client is not received within the first preset duration, the first salt-min client may have a situation such as downtime or network interruption, and further the salt-master server sets a state of the first salt-min client to an unavailable state, and prompts maintenance personnel to process in time. By setting the state of the first salt-minion client to be an unavailable state, the task is prevented from being tried to be sent to the first salt-minion client when the first salt-minion client is down or the network is interrupted, the pressure of the salt-master server is reduced, the transmission of invalid traffic in the network is reduced, and network bandwidth resources are saved.
It should be noted that the first preset time period is longer than a time interval for acquiring the system information periodically, for example, the first preset time period is 5 minutes.
According to the data processing method, system information sent by each salt-min client is obtained through a salt-master server, wherein the system information comprises CPU utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, and a monitoring analysis unit of the salt-min client regularly obtains the system information and feeds the system information back to the salt-master server; then, a monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition or not based on the system information; and if the target client which reaches the preset alarm condition exists in the salt-min client, the monitoring analysis module outputs alarm prompt information based on the target client, and the state of each salt-min client is analyzed in time through the monitoring analysis module of the salt-master server, so that maintenance personnel can be notified in time when the salt-min client is down or disconnected from the network, and the reliability of the system is improved.
Based on the first embodiment, a second embodiment of the data processing method of the present invention is proposed, in which step S102 includes:
step S201, if a second salt-minion client in an unavailable state exists in the salt-minion clients, the monitoring analysis module sets the state of the second salt-minion client to be an on-line state;
step S202, the monitoring analysis module determines whether the salt-min client reaches a preset alarm condition based on the system information.
In this embodiment, after the salt-master server obtains the system information sent by each salt-minion client, the monitoring analysis module determines whether there is system information of a second salt-minion client in an unavailable state in the system information, further determines whether there is a second salt-minion client in an unavailable state in the salt-minion clients, and if there is a second salt-minion client in an unavailable state in the salt-minion clients, the monitoring analysis module sets the state of the second salt-minion client to be in an online state, so that the second salt-minion client is in a normal state capable of receiving a task obtaining request.
And then, the monitoring analysis module determines whether the salt-min client reaches a preset alarm condition or not based on the system information.
According to the data processing method provided by the embodiment, if a second salt-minion client in an unavailable state exists in the salt-minion client, the monitoring analysis module sets the state of the second salt-minion client to be an on-line state; and then, the monitoring analysis module of the salt-master server determines whether the salt-minion client reaches a preset alarm condition based on the system information, and the state of the second salt-minion client is set to be an on-line state so as to ensure that the second salt-minion client can normally execute tasks distributed by the salt-master server, thereby improving the resource utilization rate of the CDN.
Based on the first embodiment, a third embodiment of the data processing method of the present invention is proposed, and in this embodiment, the data processing method further includes:
step S301, a content management module of a salt-master server receives a task request sent by a management platform of the salt-master server, wherein the task request comprises a client name, a content management type and a resource address;
step S302, the content management module determines a target salt-minion client corresponding to the task request in each salt-minion client based on the state information of the salt-minion client;
step S303, the content management module sends a target task to the target salt-min client based on the content management type and the resource address, wherein when the target task is received, the target salt-min client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
step S304, the management platform displays the operation result.
The salt-master server is provided with a content management module and a management platform, and the salt-minion client is provided with a content management unit.
In this embodiment, the management platform performs task scheduling, specifically, the management platform obtains a client name, a content management type and a resource address of a current task, generates and sends a task request to the content management module, and the content management module receives the task request sent by the management platform of the salt-master server, where the task request includes the content management type and the resource address. The content management module determines a target salt-minion client corresponding to the task request in each salt-minion client based on the state information of the salt-minion client, specifically, the content management module acquires the state information of each salt-minion client, acquires the salt-minion client in an online state based on the state information, and determines the target client corresponding to the task request in the salt-minion client in the online state.
Then, the content management module sends a target task to the target salt-minion client based on the content management type and the resource address, wherein when the target task is received, the target salt-minion client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server; and the management platform displays the operation result.
According to the data processing method provided by the embodiment, a task request sent by a management platform of a salt-master server is received through a content management module of the salt-master server, wherein the task request comprises a content management type and a resource address; the content management module determines a target salt-minion client corresponding to the task request from all salt-minion clients based on the state information of the salt-minion clients; the content management module sends a target task to the target salt-minion client based on the content management type and the resource address, wherein when the target task is received, the target salt-minion client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server; and then the management platform displays the operation result, so that the requirements of content management in the content network system are timely responded, the monitoring and management of the server state are perfected, a plurality of software systems are not required to be deployed on the server, and the daily management of operation and maintenance personnel is facilitated. The target salt-minion client is determined through the state information, so that the target task is prevented from being generated to the salt-minion client which is down or is interrupted by the network, the pressure of the salt-master server is reduced, the transmission of invalid traffic in the network is reduced, and the network bandwidth resources are saved.
Based on the third embodiment, a fourth embodiment of the data processing method of the present invention is proposed, and in this embodiment, after step S303, the data processing method further includes:
step S401, if the running result fed back by the target salt-min client is not received within a second preset time period after the target task is sent, updating retry times by the content management module;
step S402, if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send a target task to the target salt-minion client based on the content management type and the resource address;
step S403, if the retry number reaches the preset retry number, the content management module outputs alarm information.
In this embodiment, after the target task is sent, the duration after the target task is sent is accumulated, and if the running result fed back by the target salt-min client is not received within the second preset duration, the content management module updates the retry number.
Then, if the number of retries does not reach the preset number of retries, returning to execute step S303; and if the retry times reach the preset retry times, the content management module outputs alarm information and resets the retry times.
The second preset duration and the preset retry number may be set reasonably, for example, the second preset duration may be set to 5 minutes, and the preset retry number may be set to 3.
According to the data processing method provided by the embodiment, if the running result fed back by the target salt-min client is not received within the second preset time, the content management module updates the retry times; if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send a target task to the target salt-minimum client based on the content management type and the resource address; if the retry times reach the preset retry times, the content management module outputs alarm information, and the target task can be timely completed through a timeout retry mechanism, so that the running efficiency of the CDN is ensured.
Based on the above-described respective embodiments, a fifth embodiment of the data processing method of the present invention is proposed, in which, after step S103, the data processing method further includes:
step S501, the monitoring analysis module obtains status information of the salt-minion client, where the status information includes an online status and an unavailable status;
step S502, the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server;
in step S503, the management platform displays the system information and the status information of each salt-minion client.
In this embodiment, the monitoring and analyzing module obtains state information of the salt-min client, where the state information includes an online state and an unavailable state, and sends the state information and the state information of each salt-min client to the management platform of the salt-master server, and after receiving the state information and the state information of each salt-min client, the management platform displays system information and the state information of each salt-min client on the display interface, so that maintenance personnel can check the state of each salt-min client.
When the display interface displays the system information and the state information of each salt-minion client, if the salt-minion client in an unavailable state exists, the management platform can also display the unavailable time of the salt-minion client in the unavailable state.
According to the data processing method provided by the embodiment, the monitoring analysis module is used for acquiring the state information of the salt-minion client, wherein the state information comprises an online state and an unavailable state; the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server; and then the management platform displays the system information and the state information of each salt-minion client, so that maintenance personnel can conveniently check the information of each salt-minion client, further the management of the salt-minion client is facilitated, and the management and maintenance efficiency of the salt-minion client in the CDN is improved.
The present invention also provides a data processing apparatus, referring to fig. 3, comprising:
the acquisition module 10 is configured to acquire system information sent by each salt-minion client, where the system information includes a CPU usage rate, a memory usage rate, a disk usage amount, and a network card bandwidth, and a monitoring analysis unit of the salt-minion client periodically acquires the system information and feeds back the system information;
the monitoring analysis module 20 is configured to determine, based on the system information, whether the salt-min client reaches a preset alarm condition;
and the output module 30 is configured to output alarm prompt information based on a target client if it is determined that the target client reaching a preset alarm condition exists in the salt-minimum client.
The method executed by each program unit may refer to each embodiment of the data processing method of the present invention, and will not be described herein.
The invention also provides a computer readable storage medium.
The computer-readable storage medium of the present invention has stored thereon a data processing program which, when executed by a processor, implements the steps of the data processing method as described above.
The method implemented when the data processing program running on the processor is executed may refer to various embodiments of the data processing method of the present invention, which are not described herein again.
Furthermore, an embodiment of the present invention proposes a computer program product comprising a data processing program thereon, which, when executed by a processor, implements the steps of the data processing method as described above.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (9)
1. A data processing method, characterized in that the data processing method comprises the steps of:
the method comprises the steps that a salt-master server side obtains system information sent by each salt-master client side, wherein the system information comprises CPU (central processing unit) utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, a monitoring analysis unit of the salt-master client side obtains the system information at regular time, and feeds back the system information to the salt-master server side;
the monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition or not based on the system information;
if the fact that the target client which reaches the preset alarm condition exists in the salt-minion client is determined, the monitoring analysis module outputs alarm prompt information based on the target client;
the method comprises the steps that a content management module of a salt-master server receives a task request sent by a management platform of the salt-master server, wherein the task request comprises a content management type and a resource address;
the content management module determines a target salt-minion client corresponding to the task request from each salt-minion client based on the state information of the salt-minion client;
the content management module sends a target task to the target salt-minion client based on the content management type and the resource address, wherein when the target task is received, the target salt-minion client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
and the management platform displays the operation result.
2. The data processing method of claim 1, wherein the monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition based on the system information, comprising:
the monitoring analysis module of the salt-master server determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth;
and if the CPU utilization rate reaches the preset CPU utilization rate, the memory utilization rate reaches the preset memory utilization rate, the disk utilization amount reaches the preset disk utilization amount or the network card bandwidth reaches the preset bandwidth, the monitoring analysis module of the salt-master server determines that the salt-minion client reaches the preset alarm condition.
3. The data processing method according to claim 1, wherein the data processing method further comprises:
if the system information sent by the first salt-min client is not received within the first preset time, the salt-master server sets the state of the first salt-min client as an unavailable state.
4. The data processing method of claim 1, wherein the monitoring analysis module of the salt-master server determines whether the salt-min client reaches a preset alarm condition based on the system information, comprising:
if a second salt-minion client in an unavailable state exists in the salt-minion clients, the monitoring analysis module sets the state of the second salt-minion client to be an on-line state;
and the monitoring analysis module determines whether the salt-min client reaches a preset alarm condition or not based on the system information.
5. The data processing method of claim 1, wherein the content management module, after the step of sending the target task to the target salt-min client based on the content management type and the resource address, further comprises:
if the running result fed back by the target salt-minion client is not received within a second preset time period, the content management module updates retry times;
if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send a target task to the target salt-minimum client based on the content management type and the resource address;
and if the retry times reach the preset retry times, the content management module outputs alarm information.
6. The method for processing data according to any one of claims 1 to 5, wherein after the step of outputting the alarm prompt information by the monitoring and analyzing module if it is determined that the salt-min client reaches the preset alarm condition, further comprises:
the monitoring analysis module acquires state information of the salt-minion client, wherein the state information comprises an online state and an unavailable state;
the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server;
and the management platform displays the system information and the state information of each salt-minion client.
7. A data processing device, applied to a salt-master server, comprising:
the system comprises an acquisition module, a monitoring analysis unit and a feedback module, wherein the acquisition module is used for acquiring system information sent by each salt-minion client, wherein the system information comprises CPU (central processing unit) utilization rate, memory utilization rate, disk utilization amount and network card bandwidth, and a monitoring analysis unit of the salt-minion client acquires the system information at regular time and feeds back the system information;
the monitoring analysis module is used for determining whether the salt-min client reaches a preset alarm condition or not based on the system information;
the output module is used for outputting alarm prompt information based on a target client which reaches a preset alarm condition if the target client is determined to exist in the salt-minimum client;
the content management module is used for receiving a task request sent by a management platform of the salt-master server, wherein the task request comprises a content management type and a resource address; the content management module determines a target salt-minion client corresponding to the task request from each salt-minion client based on the state information of the salt-minion client; the content management module sends a target task to the target salt-minion client based on the content management type and the resource address, wherein when the target task is received, the target salt-minion client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
and the management platform displays the operation result.
8. A data processing apparatus, characterized in that the data processing apparatus comprises: memory, a processor and a data processing program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the data processing method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a data processing program which, when executed by a processor, implements the steps of the data processing method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110816540.7A CN113542424B (en) | 2021-07-19 | 2021-07-19 | Data processing method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110816540.7A CN113542424B (en) | 2021-07-19 | 2021-07-19 | Data processing method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113542424A CN113542424A (en) | 2021-10-22 |
| CN113542424B true CN113542424B (en) | 2023-04-28 |
Family
ID=78100287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110816540.7A Active CN113542424B (en) | 2021-07-19 | 2021-07-19 | Data processing method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113542424B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114584600A (en) * | 2022-01-20 | 2022-06-03 | 国网青海省电力公司 | Data audit monitoring system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110636088A (en) * | 2018-06-21 | 2019-12-31 | 视联动力信息技术股份有限公司 | Data monitoring method, data transmission method, device and equipment based on video link |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101741885A (en) * | 2008-11-19 | 2010-06-16 | 珠海市西山居软件有限公司 | Distributed system and method for processing task flow thereof |
| CN106844137B (en) * | 2016-12-08 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Server monitoring method and device |
| US11252689B2 (en) * | 2019-07-12 | 2022-02-15 | Charter Communications Operating, Llc | Wi-fi access point coordinated transmission of data |
| CN110389822A (en) * | 2019-07-29 | 2019-10-29 | 北京金山云网络技术有限公司 | The node scheduling method, apparatus and server of execution task |
| CN111611131B (en) * | 2020-05-18 | 2023-06-30 | 中移（杭州）信息技术有限公司 | Operation and maintenance method, device and system based on Saltstack and storage medium |
-
2021
- 2021-07-19 CN CN202110816540.7A patent/CN113542424B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110636088A (en) * | 2018-06-21 | 2019-12-31 | 视联动力信息技术股份有限公司 | Data monitoring method, data transmission method, device and equipment based on video link |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113542424A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107645529B (en) | Heartbeat packet sending method and device | |
| CN109510723B (en) | Gateway equipment, and transaction management and control system and method of Internet of things | |
| US8930521B2 (en) | Method, apparatus, and computer program product for enabling monitoring of a resource | |
| US9001694B2 (en) | Methods and apparatus for controlling a wireless station | |
| CN108200132B (en) | Resource acquisition method, device, equipment and computer readable storage medium | |
| CN107908487B (en) | Task control management method, device and equipment and computer readable storage medium | |
| CN113900834B (en) | Data processing method, device, equipment and storage medium based on Internet of things technology | |
| CN105262608A (en) | Monitoring method and monitoring device for network service | |
| CN114024972A (en) | Long connection communication method, system, device, equipment and storage medium | |
| CN109428926B (en) | Method and device for scheduling task nodes | |
| CN109039899B (en) | Data transmission method, system and computer readable storage medium | |
| CN113542424B (en) | Data processing method, device, equipment and readable storage medium | |
| CN112052133A (en) | Service system monitoring method and device based on Kubernetes | |
| CN113162848A (en) | Method, device, gateway and medium for realizing block chain gateway | |
| CN117608825A (en) | Resource management method based on multi-cloud management platform and related equipment | |
| CN113885794B (en) | Data access method and device based on multi-cloud storage, computer equipment and medium | |
| US10432490B2 (en) | Monitoring single content page application transitions | |
| EP3373608B1 (en) | Method and apparatus for acquiring resources | |
| CN111831503B (en) | Monitoring method based on monitoring agent and monitoring agent device | |
| CN106657195B (en) | Task processing method and relay device | |
| US10616081B2 (en) | Application aware cluster monitoring | |
| CN115981871A (en) | GPU resource scheduling method, device, equipment and storage medium | |
| JP2014041404A (en) | Terminal service monitoring device | |
| CN111726417B (en) | Delay control method, device, server and storage medium | |
| CN106406921B (en) | Method and device for installing patch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |