CN112838941B - Data processing method, device, system and storage medium - Google Patents
Data processing method, device, system and storage medium Download PDFInfo
- Publication number
- CN112838941B CN112838941B CN201911167301.2A CN201911167301A CN112838941B CN 112838941 B CN112838941 B CN 112838941B CN 201911167301 A CN201911167301 A CN 201911167301A CN 112838941 B CN112838941 B CN 112838941B
- Authority
- CN
- China
- Prior art keywords
- service
- data processing
- access
- inter
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003860 storage Methods 0.000 title claims abstract description 19
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 142
- 238000012423 maintenance Methods 0.000 claims abstract description 37
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000007405 data analysis Methods 0.000 claims description 22
- 238000004891 communication Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 17
- 238000012544 monitoring process Methods 0.000 claims description 16
- 230000000977 initiatory effect Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 13
- 230000009286 beneficial effect Effects 0.000 abstract description 5
- 230000003993 interaction Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- 230000003068 static effect Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/5058—Service discovery by the service manager
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the application provides a data processing method, device, system and storage medium. In the data processing system, based on the interaction between the edge node and the data processing node, when the inter-service access operation exists in the data processing system, the association relation of different services in the data processing system can be obtained according to the message corresponding to the inter-service access operation. The association relation is dynamically obtained according to the actual running process of the data processing system, can accurately reflect the dependency relation of different services when the data processing system is actually running, and is beneficial to effectively guiding and early warning the operation and maintenance operation of the data processing system.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method, device, system, and storage medium.
Background
The cloud computing environment integrates resources such as network, computing, storage and the like into a unified and operable system through distributed deployment. In the process of operating and maintaining the cloud data processing system with high integration level, the dependency relationship among various cloud products, components or cloud services in the cloud data processing system is required to be acquired.
In the prior art, the obtained dependency relationship between different cloud services is usually a cloud service static dependency relationship in design. The static dependency relationship cannot accurately reflect the dependency relationship of the cloud service in operation, and is unfavorable for the operation and maintenance of the cloud data processing system. Therefore, a solution is needed.
Disclosure of Invention
Aspects of the present application provide a data processing method, apparatus, system, and storage medium for obtaining a more accurate service dependency relationship.
An embodiment of the present application provides a data processing system, including: an edge node and a data processing node; wherein the edge node is configured to: acquiring access data corresponding to at least one inter-service access operation in a data processing system and providing the access data to the data processing nodes; the data processing node is configured to: and acquiring at least one group of services with access dependency relationships according to the access data corresponding to the access operations among the at least one service, and acquiring service association relationships in the data processing system according to the at least one group of services with access dependency relationships.
The embodiment of the application also provides a data processing method, which comprises the following steps: acquiring access data corresponding to at least one inter-service access operation in a data processing system; acquiring at least one group of services with access dependency relationships according to access data corresponding to each access operation among the at least one service; and acquiring service association relations in the data processing system according to the at least one group of services with access dependency relations.
The embodiment of the application also provides a data processing device, which comprises: a memory, a processor, and a communication component; the memory is used for storing a computer program; the processor is coupled with the memory and the communication component, and is used for executing a computer program for steps or operations in the data processing method provided by the embodiment of the application.
The embodiment of the application also provides a computer readable storage medium storing a computer program, which is characterized in that the computer program can realize steps or operations in the data processing method provided by the embodiment of the application when being executed by a computer.
In the data processing system provided by the embodiment of the application, based on the interaction between the edge node and the data processing node, when the inter-service access operation exists in the data processing system, the association relationship of different services in the data processing system can be obtained according to the message corresponding to the inter-service access operation. The association relation is dynamically obtained according to the actual running process of the data processing system, can accurately reflect the dependency relation of different services when the data processing system is actually running, and is beneficial to effectively guiding and early warning the operation and maintenance operation of the data processing system.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1a is a schematic diagram of a data processing system according to an exemplary embodiment of the present application;
FIG. 1b is a schematic diagram of a data processing system according to another exemplary embodiment of the present application;
FIG. 2a is a schematic illustration of obtaining a service with an access dependency according to a message according to an embodiment of the present application;
FIG. 2b is a schematic representation of a service relationship network topology provided by an exemplary embodiment of the present application;
FIG. 3a is a schematic diagram of a data processing system according to an embodiment of the present application;
FIG. 3b is a schematic representation of a service relationship network topology provided by an application scenario embodiment of the present application;
FIG. 4 is a flow chart of a data processing method according to an exemplary embodiment of the present application;
Fig. 5 is a schematic diagram of a data processing apparatus according to an exemplary embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Aiming at the technical problem that the dependency relationship of different service operation in the data processing system cannot be accurately obtained in the prior art, a solution is provided, and the technical scheme provided by the embodiments of the application is described in detail below with reference to the accompanying drawings.
FIG. 1a is a schematic diagram of a data processing system according to an exemplary embodiment of the present application, and as shown in FIG. 1a, the data processing system 100 includes: edge node 10 and data processing node 20.
In data processing system 100, edge node 10 is primarily used to: access data corresponding to each of at least one inter-service access operation in the data processing system is obtained and provided to data processing node 20. In some embodiments, the edge node 10 may be implemented as a switch at an edge layer of a network, such as an ethernet switch, a fiber switch, or the like, and the present embodiment is not limited.
Where a data processing system refers to a system that can provide computing, storage, analysis, querying, and network access capabilities to users, devices, or processes based on hardware resources, software resources, and network resources. A data processing system may be comprised of one or more computing devices. In some embodiments, the data processing system may be implemented as a cloud computing system, a distributed system, a content distribution system, a parallel data processing system, a network storage system, a big data system, etc., including but not limited to. Different data processing systems may provide a wide variety of different services to achieve particular data processing functions.
Inter-service access operations refer to inter-access operations between different services provided by a data processing system, such as operations of a data query service accessing a data storage service and operations of a data analysis service accessing a network service. The inter-service access operation may generate corresponding access data that may be forwarded by edge node 20 to effect the access operation as it passes through edge node 20.
In some embodiments, edge node 20 may provide access data corresponding to inter-service access operations to data processing node 20 on the fly each time that access data is acquired; the set period of time may be used as a data providing period, and the access data corresponding to the inter-service access operation obtained by accumulating in the set period of time may be provided to the data processing node 20, which is not limited in this embodiment.
In data processing system 100, data processing node 20 may be implemented as any device or devices having data processing and communication functions, as the embodiment is not limited. For example, in some embodiments, data processing node 20 may be implemented as a server, including conventional servers, cloud hosts, virtual centers, etc., including but not limited to.
After receiving the access data sent by the edge node 10, the data processing node 20 can obtain at least one group of services with access dependency relationship according to the access data sent by the edge node 10; based on the at least one set of services having access dependencies, data processing node 20 may obtain service associations in the data processing system.
Wherein a set of services that have access dependencies may include two or more numbers of services. Taking any two services with access dependency as an example, the two services with access dependency can be expressed as: the implementation process of one service needs to provide support for the other service, or the execution result of one service can affect the operation of the other service, or can also be represented by other dependency forms, which are not described herein.
In the data processing system 100, in order to implement the above-mentioned data interaction process between the monitoring sub-node 10 and the data processing node 20, the monitoring sub-node 10 and the data processing node 20 may establish a communication connection, and a specific communication connection manner may depend on an actual application scenario.
In this embodiment, based on the interaction between the edge node 10 and the data processing node 20 in the data processing system 100, when there is an inter-service access operation in the data processing system, the association relationship of different services in the data processing system can be obtained according to the access data corresponding to the inter-service access operation. The association relation is dynamically obtained according to the actual running process of the data processing system, can accurately reflect the dependency relation of different services when the data processing system is actually running, and is beneficial to effectively guiding and early warning the operation and maintenance operation of the data processing system.
In some exemplary embodiments, data processing node 20 includes: monitor child node 201 and data analysis child node 202. The monitoring sub-node 201 and the data analysis sub-node 202 may be deployed on one device, or may be deployed on different devices as shown in fig. 1b, which is not limited in this embodiment.
In the data processing system 100, the edge node 10 and the monitoring sub-node 201 may establish a communication connection relationship based on a wired communication or a wireless communication manner, and further, the edge node 10 may provide the obtained access data corresponding to each of the at least one inter-service access operation to the monitoring sub-node 201.
Alternatively, the access data corresponding to the inter-service access operation may be implemented as a message generated by the inter-service access operation, and the edge node 10 may acquire a message generated by at least one inter-service access operation and synchronize a message image generated by the at least one inter-service access operation to the monitoring child node 201.
Optionally, when the edge node 10 is implemented as a switch, an RSPAN (Remote Local Switched Port Analyzer, remote switch port analyzer, a port mirroring technique of the switch) may be used to obtain all the messages passing through the edge node 10, generate a copy of the message, and send the copy of the message to the monitoring child node 201.
Based on this way of mirror synchronizing the message copies, the data exchange operation of the edge node 10 is not affected, and the edge node can mirror all the messages to the monitoring child node 201 while guaranteeing the original traffic and providing quality of service (Quality of Service, qos) capability.
After the monitoring sub-node 201 obtains the packet sent by the edge node 10, service identification information corresponding to each of the at least one inter-service access operation may be extracted from the packet, and the service identification information corresponding to each of the at least one inter-service access operation may be sent to the data analysis sub-node 202. The service identification information can uniquely identify the service provided by the data processing system, and based on the service identification information, the service related to the access operation among the services can be determined.
After receiving the service identification information corresponding to each of the at least one inter-service access operation, the data analysis sub-node 202 may identify at least one group of services having an access dependency relationship according to the service identification information corresponding to each of the at least one inter-service access operation, and calculate a service association relationship in the data processing system according to the at least one group of services having an access dependency relationship.
In some alternative examples, the service identification information may be implemented as network element group information corresponding to the service. Optionally, the network element group information may include a Port (Port) and an IP (nternet Protocol, internet protocol) address of the device where the service is located.
Based on this, in data processing system 100, monitoring child node 201 may be specifically configured to: for any inter-service access operation in the data processing system, a source IP address, a source port, a destination IP address, and a destination port associated with the inter-service access operation are extracted from a message corresponding to the inter-service access operation, and are used as service identification information corresponding to the inter-service access operation, and are sent to the data analysis subnode 202. A typical structure of a message corresponding to an inter-service access operation may be shown in FIG. 2a, where according to definitions of different fields in the message, it may be determined that an IP address included in the message is a source IP address or a destination IP address, and it may be determined that a port included in the message is a source port or a destination port.
After receiving the service identification information, the data analysis sub-node 202 may determine, according to a source IP address and a source port included in the service identification information, a source service that initiates the inter-service access operation, determine, according to a destination IP address and a destination port included in the service identification information, a destination service that is requested to be accessed by the inter-service access operation, and use the source service and the destination service as a group of services that have access dependency relationships. It should be noted that, the correspondence between the service provided by the data processing system and the IP address and the port may be pre-established and stored at the data analysis sub-node 202 for subsequent use, which is not described herein.
As shown in fig. 2a, according to the message format and the definition of different fields in the message, it may be determined that the destination IP included in the message is IPA, the destination port is PortA, the source IP is IPB, and the source port is PortB. Based on the corresponding relation between the cloud service A and IPA+ PortA, the cloud service A can be determined to be an initiator of the inter-service access operation, and based on the corresponding relation between the cloud service B and IPB+ PortB, the cloud service B can be determined to be a receiver of the inter-service access operation.
Based on the implementation manner described in the foregoing embodiment, the data analysis sub-node 202 may obtain multiple sets of access dependency relationships on the data processing system, and analyze and obtain a total service association relationship corresponding to the data processing system through the multiple sets of access dependency relationships. For ease of description, services in a set of access dependencies are described as a service group. For any service on the data system, the data analysis sub-node 202 may obtain all service groups containing the service; based on all of the service groups that contain the service, data analysis sub-node 202 may obtain all other services on the data processing system that have a dependency relationship with the service. When other services with dependency relations corresponding to each service are obtained, the full service association relations corresponding to the data processing system can be obtained.
In some illustrative embodiments, data processing system 100 also includes: the node 30 is shown. The data analysis sub-node 202 may draw a service relationship network topology according to the calculated service association relationship, and send the service relationship network topology to the display node 30. The display node 30 may present the service relationship network topology upon receiving the service relationship network topology.
Alternatively, when the data analysis sub-node 202 draws the service relationship network topology graph, each service may be used as a topology node in the service relationship network topology graph, and a topology line segment is used to connect services having a dependency relationship, as shown in fig. 2 b. Optionally, as shown in fig. 2b, the display node 30 may also present the IP and port corresponding to each service for user viewing.
In some exemplary embodiments, data processing node 20 further comprises: decision child node 203. The data analysis sub-node 202 and the decision sub-node 203 may be deployed on the same device, implemented as two applications on the device, or implemented as two different modules on an application. Or the data analysis sub-node 202 and the decision sub-node 203 may be deployed on different devices, as shown in fig. 1b, which is not limiting in this embodiment.
Alternatively, the data analysis sub-node 202 may send the acquired service association relationship in the data processing system to the decision sub-node 303. The decision sub-node 303 may predict operation and maintenance impact data of any service in the data processing system according to the service association relationship, where the operation and maintenance impact data includes the impact degree of operation and maintenance operation of the service on other services, and send the impact degree to the display node 30. Further, display node 30 may highlight the extent to which the operation of any one service contained in the data processing system affects other services on the service relationship network topology. Alternatively, the manner of highlighting the influence degree may include a chart manner, a text manner, a highlighting manner with different colors, etc., which is not limited in this embodiment.
For example, a service relationship network topology corresponding to a data processing system may be as shown in fig. 2b, where the service directly connected to the service a is: the service B and the service C can be considered as stronger access dependency relationship between the service B and the service A and between the service C and the service A; the services indirectly connected with the service A are as follows: service D and service E may be considered as having weak access dependencies between service D and E and service a. Then, when performing an operation on service a, the decision sub-node 303 may consider that service B, service C will be affected more and service D and service E will be affected less. At this time, the service B, the service C, the service D, and the service E may be highlighted with different colors, for example, the service B, the service C may be highlighted with red of high saturation, and the service D and the service E may be highlighted with red of low saturation.
In the following, with reference to fig. 3a and fig. 3b, an application scenario in which a data processing system is implemented as a cloud computing system will be taken as an example to describe an exemplary embodiment of the present application.
Multiple cloud products in a cloud computing system may provide a variety of different cloud services. When the cloud computing system is designed, setting corresponding network element group information for each cloud service, wherein the network element group information comprises the IP of equipment where the cloud service is located and a port corresponding to the cloud service. The correspondence between each cloud service and the network element group information may be stored in a server.
As shown in fig. 3a, when an inter-cloud service access operation occurs in the cloud computing system, the edge node may remotely mirror all the messages passing through the switch to the monitoring child node through the RSPAN. Assume that the switch detects a message that one cloud service accesses another cloud service and mirrors the detected message to the monitoring child node. After the monitoring sub-node receives the message, multiple groups of network element group information can be extracted from the message and sent to the data analysis sub-node. After receiving the multiple sets of network element group information, the data analysis sub-node can identify cloud services corresponding to the multiple sets of network element group information based on the corresponding relation between the cloud services and the network element group information, which are stored in advance. Assuming that cloud service SR1 accesses cloud service SR2, cloud service SR1 and cloud service SR2 may be determined to be a set of cloud services that have access dependencies. And after acquiring the total cloud service dependency relationship, transmitting the total cloud service dependency relationship to the decision sub-node. The decision-making sub-node can respond to operation and maintenance changing operation of the cloud service, analyze operation and maintenance influence data and send the operation and maintenance influence data to the display node. The display node can draw a service relation network topological graph of the cloud computing system, and when a certain service operates and maintains, other affected services are highlighted. Assume that a service relationship network topology of a cloud computing system is shown in fig. 3 b. If in the cloud computing system, the cloud service SR1 is upgraded and operated. During the upgrade of SR1, SR1 will execute the restarting operation and its configuration will change.
Based on the service association relationship, the decision-making sub-node can determine that the cloud service SR2 directly depends on the service of the cloud service SR1, and the cloud service SR3 depends on the service of the cloud service SR2, so that the upgrade operation and maintenance operation of the cloud service SR1 may directly affect the normal service of the cloud service SR2 and indirectly affect the normal service of the cloud service SR 3. The decision-making sub-node can send the influence result to the display node, and the display node can carry out distinguishing highlighting on the cloud service SR2 and the cloud service SR3 on the service relation network topological graph so as to assist operation and maintenance personnel in analyzing the operation and maintenance influence.
In addition to the data processing system provided in the foregoing embodiments, an embodiment of the present application further provides a data processing method, which will be described below with reference to the accompanying drawings.
FIG. 4 is a flowchart of a data processing method according to an exemplary embodiment of the present application, which may be performed by the data processing system according to the foregoing embodiment, and may include the steps shown in FIG. 4:
Step 401, access data corresponding to each access operation between at least one service in the data processing system is obtained.
Step 402, obtaining at least one group of services with access dependency relationships according to the access data corresponding to the access operations among the at least one service.
Step 403, obtaining a service association relationship in the data processing system according to the at least one group of services with access dependency relationship.
In some exemplary embodiments, according to the access data corresponding to each of the at least one inter-service access operation, a manner of obtaining at least one set of services having an access dependency relationship includes: extracting service identification information corresponding to each of the at least one inter-service access operation from the message corresponding to each of the at least one inter-service access operation; identifying at least one group of services with access dependency relationships according to the service identification information corresponding to each access operation among the at least one service; and calculating service association relations in the data processing system according to the at least one group of services with access dependency relations.
In some exemplary embodiments, one way of extracting service identification information corresponding to each of the at least one inter-service access operation from the message corresponding to each of the at least one inter-service access operation includes: and extracting network element group information associated with the inter-service access operation from a message corresponding to the inter-service access operation aiming at any one of the at least one inter-service access operation.
In some exemplary embodiments, one way of identifying at least one group of services having an access dependency relationship according to the service identification information corresponding to each of the at least one inter-service access operation includes: determining a source service initiating the inter-service access operation and a destination service requested to be accessed by the inter-service access operation according to the network element group information; and the source service and the destination source service are used as a group of services with access dependency relations.
In some exemplary embodiments, the network element group information includes: source IP address, source port, destination IP address, and destination port.
In some exemplary embodiments, the method further comprises: and drawing a service relationship network topology graph according to the service association relationship in the data processing system, and displaying the service relationship network topology graph.
In some exemplary embodiments, the method further comprises: predicting operation and maintenance influence data of any service in the data processing system according to the service association relation in the data processing system; the operation and maintenance influence data comprises the influence degree of operation and maintenance operation of the service on other services; on the service relation network topological graph, the operation and maintenance influence data of any service contained in the data processing system is highlighted; the operation and maintenance influence data comprises influence degrees of operation and maintenance operation of the service on other services.
In this embodiment, when an inter-service access operation exists in the data processing system, the association relationship of different services in the data processing system is obtained according to the message corresponding to the inter-service access operation. The association relation is dynamically obtained according to the actual running process of the data processing system, can accurately reflect the dependency relation of different services when the data processing system is actually running, and is beneficial to effectively guiding and early warning the operation and maintenance operation of the data processing system.
It should be noted that, the execution subjects of each step of the method provided in the above embodiment may be the same device, or the method may also be executed by different devices. For example, the execution subject of steps 401 to 403 may be device a; for another example, the execution subject of steps 401 and 402 may be device a, and the execution subject of step 403 may be device B; etc.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that the operations may be performed out of the order in which they appear herein or performed in parallel, the sequence numbers of the operations, such as 401, 402, etc., are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.
Fig. 5 is a schematic diagram of a data processing apparatus according to an exemplary embodiment of the present application, which is applicable to the data processing system according to the foregoing embodiment. As shown in fig. 5, the data processing apparatus includes: memory 501, processor 502, and communication component 503.
Memory 501 is used to store computer programs and may be configured to store various other data to support operations on the data processing device. Examples of such data include instructions for any application or method operating on the data processing device, contact data, phonebook data, messages, pictures, video, and the like.
The memory 501 may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
A processor 502 coupled to the memory 501 for executing the computer program in the memory 501 for: acquiring access data corresponding to at least one inter-service access operation in a data processing system; acquiring at least one group of services with access dependency relationships according to access data corresponding to each access operation among the at least one service; and acquiring service association relations in the data processing system according to the at least one group of services with access dependency relations.
Further optionally, when the processor 502 obtains at least one group of services with access dependency relationships according to the access data corresponding to the access operation between the at least one service, the processor is specifically configured to: extracting service identification information corresponding to each of the at least one inter-service access operation from the message corresponding to each of the at least one inter-service access operation; identifying at least one group of services with access dependency relationships according to the service identification information corresponding to each access operation among the at least one service; and calculating service association relations in the data processing system according to the at least one group of services with access dependency relations.
Further optionally, the processor 502 is specifically configured to, when extracting service identification information corresponding to each of the at least one inter-service access operation from a packet corresponding to each of the at least one inter-service access operation: and extracting network element group information associated with the inter-service access operation from a message corresponding to the inter-service access operation aiming at any one of the at least one inter-service access operation.
Further optionally, the processor 502 is specifically configured to, when identifying at least one group of services having an access dependency relationship according to the service identification information corresponding to each of the at least one inter-service access operation: determining a source service initiating the inter-service access operation and a destination service requested to be accessed by the inter-service access operation according to the network element group information; and the source service and the destination source service are used as a group of services with access dependency relations.
Further optionally, the network element group information includes: source IP address, source port, destination IP address, and destination port.
Further optionally, the processor 502 is further configured to: and drawing a service relationship network topology graph according to the service association relationship in the data processing system, and displaying the service relationship network topology graph.
Further optionally, the processor 502 is further configured to: predicting operation and maintenance influence data of any service in the data processing system according to the service association relation in the data processing system; the operation and maintenance influence data comprises the influence degree of operation and maintenance operation of the service on other services; on the service relation network topological graph, the operation and maintenance influence data of any service contained in the data processing system is highlighted; the operation and maintenance influence data comprises influence degrees of operation and maintenance operation of the service on other services.
Further, as shown in fig. 5, the data processing apparatus further includes: a display 504, a power supply component 505, an audio component 506, and other components. Only part of the components are schematically shown in fig. 5, which does not mean that the data processing device only comprises the components shown in fig. 5.
Wherein the communication component is configured to facilitate wired or wireless communication between the device in which the communication component is located and other devices. The device in which the communication component is located may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC) technology, radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
Among them, the display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.
The power supply component provides power for various components of equipment where the power supply component is located. The power components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the devices in which the power components are located.
In this embodiment, when an inter-service access operation exists in the data processing system, the association relationship of different services in the data processing system is obtained according to the message corresponding to the inter-service access operation. The association relation is dynamically obtained according to the actual running process of the data processing system, can accurately reflect the dependency relation of different services when the data processing system is actually running, and is beneficial to effectively guiding and early warning the operation and maintenance operation of the data processing system.
Accordingly, embodiments of the present application also provide a computer-readable storage medium storing a computer program which, when executed, is capable of implementing the steps of the above-described method embodiments that are executable by a data processing apparatus.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (12)
1. A data processing system, comprising: an edge node and a data processing node; the edge node is a switch of a network edge layer; the data processing node comprises: monitoring the child node and analyzing the data by the child node;
The edge node is configured to: acquiring access data corresponding to at least one inter-service access operation in a data processing system and providing the access data to the data processing nodes; the access data corresponding to any service-to-service access operation comprises: the message generated by the inter-service access operation; the edge node is specifically configured to: synchronizing the obtained message mirror image generated by the at least one inter-service access operation to the monitoring child node;
The data processing node is configured to: acquiring at least one group of services with access dependency relationships according to access data corresponding to each access operation among the at least one service, and acquiring service association relationships in the data processing system according to the at least one group of services with access dependency relationships;
The data processing node is specifically configured to: extracting service identification information corresponding to each of the at least one inter-service access operation from the message corresponding to each of the at least one inter-service access operation by using the monitoring sub-node, and sending the service identification information to the data analysis sub-node; the service identification information corresponding to any inter-service access operation is network element group information corresponding to the inter-service access operation;
And identifying at least one group of services with access dependency relationships according to the service identification information corresponding to each access operation among the at least one service by utilizing the data analysis sub-node, and calculating service association relationships in the data processing system according to the at least one group of services with access dependency relationships.
2. The system according to claim 1, wherein the data analysis sub-node is specifically configured to:
and determining a source service initiating the inter-service access operation and a destination service requested to be accessed by the inter-service access operation as a group of services with access dependency relations according to the network element group information.
3. The system according to claim 1 or 2, wherein the network element group information comprises: source IP address, source port, destination IP address, and destination port.
4. The system of any one of claims 1-2, further comprising: displaying the nodes;
The data analysis child node is further configured to: drawing a service relation network topology graph according to the service association relation, and sending the service relation network topology graph to the display node;
And the display node is used for displaying the service relation network topological graph.
5. The system of claim 4, wherein the data processing node further comprises: decision-making sub-nodes;
The decision sub-node is configured to: predicting operation and maintenance influence data of any service in the data processing system according to the service association relation in the data processing system; the operation and maintenance influence data comprises the influence degree of operation and maintenance operation of the service on other services;
The display node is further configured to: operation and maintenance influence data of any service contained in the data processing system are highlighted on the service relation network topological graph.
6. A method of data processing, comprising:
The data node acquires access data corresponding to each access operation between at least one service in the data processing system; the access data corresponding to any service-to-service access operation comprises: the message generated by the inter-service access operation; obtaining access data corresponding to at least one inter-service access operation in the data processing system, wherein the access data comprises: acquiring a message generated by the at least one inter-service access operation of the edge node mirror synchronization; the edge node is a switch of a network edge layer;
Acquiring at least one group of services with access dependency relationships according to access data corresponding to each access operation among the at least one service;
acquiring service association relations in the data processing system according to the at least one group of services with access dependency relations;
The method for acquiring at least one group of services with access dependency relationship according to the access data corresponding to the access operation among the at least one service comprises the following steps: extracting service identification information corresponding to each of the at least one inter-service access operation from the message corresponding to each of the at least one inter-service access operation; the service identification information corresponding to any inter-service access operation is network element group information corresponding to the inter-service access operation; identifying at least one group of services with access dependency relationships according to the service identification information corresponding to each access operation among the at least one service; the service identification information corresponding to any inter-service access operation is the network element group information corresponding to the inter-service access operation.
7. The method of claim 6, wherein identifying at least one group of services for which an access dependency exists based on the service identification information corresponding to each of the at least one inter-service access operation, comprises:
Determining a source service initiating the inter-service access operation and a destination service requested to be accessed by the inter-service access operation according to the network element group information;
and the source service and the destination service are used as a group of services with access dependency relations.
8. The method of claim 6, wherein the network element group information comprises: source IP address, source port, destination IP address, and destination port.
9. The method according to any one of claims 7-8, further comprising:
and drawing a service relationship network topology graph according to the service association relationship in the data processing system, and displaying the service relationship network topology graph.
10. The method as recited in claim 9, further comprising:
Predicting operation and maintenance influence data of any service in the data processing system according to the service association relation in the data processing system; the operation and maintenance influence data comprises the influence degree of operation and maintenance operation of the service on other services;
On the service relation network topological graph, the operation and maintenance influence data of any service contained in the data processing system is highlighted; the operation and maintenance influence data comprises influence degrees of operation and maintenance operation of the service on other services.
11. A data processing apparatus, comprising: a memory, a processor, and a communication component;
the memory is used for storing a computer program;
The processor, coupled to the memory and the communication component, for executing a computer program for performing the steps or operations of the data processing method of any of claims 6-10.
12. A computer readable storage medium storing a computer program, which when executed by a computer is capable of carrying out the steps or operations of the data processing method of any one of claims 6 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911167301.2A CN112838941B (en) | 2019-11-25 | 2019-11-25 | Data processing method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911167301.2A CN112838941B (en) | 2019-11-25 | 2019-11-25 | Data processing method, device, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112838941A CN112838941A (en) | 2021-05-25 |
| CN112838941B true CN112838941B (en) | 2024-04-19 |
Family
ID=75922501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911167301.2A Active CN112838941B (en) | 2019-11-25 | 2019-11-25 | Data processing method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112838941B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103533023A (en) * | 2013-07-25 | 2014-01-22 | 上海和辰信息技术有限公司 | Cloud service application cluster synchronization system and synchronization method based on cloud service characteristics |
| CN104092755A (en) * | 2014-07-07 | 2014-10-08 | 武汉科技大学 | Capturing method and device for cloud service origin data |
| CN108475360A (en) * | 2015-12-31 | 2018-08-31 | 微软技术许可有限责任公司 | Distributed Calculation relies on management system |
| CN109361608A (en) * | 2018-11-23 | 2019-02-19 | 北京六方领安网络科技有限公司 | Message processing method, system and storage medium |
| CN109587255A (en) * | 2018-12-12 | 2019-04-05 | 深圳先进技术研究院 | Service building method, system and the storage medium of dependence graph |
-
2019
- 2019-11-25 CN CN201911167301.2A patent/CN112838941B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103533023A (en) * | 2013-07-25 | 2014-01-22 | 上海和辰信息技术有限公司 | Cloud service application cluster synchronization system and synchronization method based on cloud service characteristics |
| CN104092755A (en) * | 2014-07-07 | 2014-10-08 | 武汉科技大学 | Capturing method and device for cloud service origin data |
| CN108475360A (en) * | 2015-12-31 | 2018-08-31 | 微软技术许可有限责任公司 | Distributed Calculation relies on management system |
| CN109361608A (en) * | 2018-11-23 | 2019-02-19 | 北京六方领安网络科技有限公司 | Message processing method, system and storage medium |
| CN109587255A (en) * | 2018-12-12 | 2019-04-05 | 深圳先进技术研究院 | Service building method, system and the storage medium of dependence graph |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112838941A (en) | 2021-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11372669B2 (en) | System and method of cross-silo discovery and mapping of storage, hypervisors and other network objects | |
| US10484265B2 (en) | Dynamic update of virtual network topology | |
| CN113726846B (en) | Edge cloud system, resource scheduling method, equipment and storage medium | |
| US10491453B2 (en) | Correlating computing network events | |
| US9130943B1 (en) | Managing communications between client applications and application resources of on-premises and cloud computing nodes | |
| CN112069422A (en) | Information display method, server, terminal and medium | |
| US20150288619A1 (en) | Distributed service management platform | |
| US20170168881A1 (en) | Process chain discovery across communication channels | |
| US9641405B2 (en) | System and method for sequencing per-hop data in performance-monitored network environments | |
| CN109936474B (en) | Method and equipment for generating network topological graph | |
| CN112994987B (en) | Cloud network, measurement system, method, device and storage medium for cloud network | |
| CN114679416A (en) | Robot communication method, system, equipment and storage medium | |
| CN110875947A (en) | Data processing method and edge node equipment | |
| CN114513552A (en) | Data processing method, device, equipment and storage medium | |
| CN112838941B (en) | Data processing method, device, system and storage medium | |
| JP2021170319A (en) | Method and device for acquiring information | |
| WO2021097713A1 (en) | Distributed security testing system, method and device, and storage medium | |
| EP4280561A1 (en) | Information flow identification method, network chip, and network device | |
| EP4340436A1 (en) | Operation and maintenance method, apparatus, and system, server, electronic device, and medium | |
| CN115643203A (en) | Content distribution method, content distribution device, content distribution network, device, and medium | |
| CN114039860B (en) | Method and system for quickly constructing server network topology graph | |
| WO2022110974A1 (en) | Method and apparatus for training data analysis model, and storage medium | |
| CN110881064B (en) | Domain name configuration method and device | |
| CN109286532A (en) | The management method and device of warning information in cloud computing system | |
| CN111104575B (en) | Data grabbing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |