CN116248559A - Method, system, device, equipment and medium for acquiring monitoring log - Google Patents
Method, system, device, equipment and medium for acquiring monitoring log Download PDFInfo
- Publication number
- CN116248559A CN116248559A CN202310456533.XA CN202310456533A CN116248559A CN 116248559 A CN116248559 A CN 116248559A CN 202310456533 A CN202310456533 A CN 202310456533A CN 116248559 A CN116248559 A CN 116248559A
- Authority
- CN
- China
- Prior art keywords
- log
- monitoring
- virtual machine
- target
- monitoring module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 256
- 238000000034 method Methods 0.000 title claims abstract description 83
- 230000008569 process Effects 0.000 claims abstract description 21
- 238000003860 storage Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 description 8
- 238000012423 maintenance Methods 0.000 description 6
- 238000011144 upstream manufacturing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011418 maintenance treatment Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/085—Retrieval of network configuration; Tracking network configuration history
- H04L41/0853—Retrieval of network configuration; Tracking network configuration history by actively collecting configuration information or by backing up configuration information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/06—Generation of reports
- H04L43/062—Generation of reports related to network traffic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a method, a system, a device, equipment and a medium for acquiring a monitoring log, wherein the method comprises the following steps: the method comprises the steps that a log monitoring module obtains target flow data and forwards the target flow data to a first virtual machine so that the first virtual machine processes the target flow data, wherein target equipment comprises at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and the virtual machine is bound with the log monitoring module; the log monitoring module generates a monitoring log based on the target flow data. According to the method and the device, a log monitoring module can be matched with each virtual machine, so that flow bottlenecks can be reduced, all flow input into the virtual machines can be monitored, and monitoring logs can be generated efficiently and accurately.
Description
Technical Field
The embodiment of the application relates to the field of network security, in particular to a method, a system, a device, equipment and a medium for acquiring a monitoring log.
Background
The service monitoring log is an important data source for service monitoring, and the service monitoring includes, but is not limited to, interface GPS, interface delay, interface error rate, other compound service indexes, and the like, and almost all services need service monitoring to ensure the stability of a self system. In the related art, the operation of outputting the service monitoring log is put down to each service party, so that the problems of various formats, non-uniform paths and the like of the log are caused, and the generated monitoring log is inaccurate.
Therefore, how to efficiently and accurately generate the monitoring log becomes a problem to be solved.
Disclosure of Invention
The embodiment of the application provides a method, a system, a device, equipment and a medium for acquiring a monitoring log, which can be used for at least matching one log monitoring module for each virtual machine according to some embodiments of the application, so that flow bottlenecks can be reduced, all flows input into the virtual machine can be monitored, and the monitoring log can be generated efficiently and accurately.
In a first aspect, the present application provides a method for obtaining a monitoring log, applied to a target device, where the method includes: the method comprises the steps that a log monitoring module obtains target flow data and forwards the target flow data to a first virtual machine so that the first virtual machine processes the target flow data, wherein target equipment comprises at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and the virtual machine is bound with the log monitoring module; the log monitoring module generates a monitoring log based on the target flow data.
Therefore, unlike the method of using one gateway to correspond to a plurality of virtual machines in the related art, in the embodiment of the present application, by matching one log monitoring module for each virtual machine, traffic data of all access virtual machines (including traffic data of access virtual machines from clients and traffic data of access to other virtual machines) can be obtained, so that a complete monitoring log can be generated. Meanwhile, the log monitoring module in the method is only responsible for log processing of the virtual machine bound with the log monitoring module, so that IO bottleneck can not be caused, the operation efficiency of the target equipment is improved, and the monitoring log can be efficiently and accurately generated.
With reference to the first aspect, in an implementation manner of the present application, before the log monitoring module obtains the target traffic data, the method further includes: acquiring the request type of the target flow data; searching a monitoring port number corresponding to the request type in a monitoring port comparison table, wherein the monitoring port comparison table is used for storing the corresponding relation between the request type and the monitoring port number; modifying the monitoring port number of the log monitoring module; the log monitoring module obtains target flow data, including: and the log monitoring module acquires the target flow data through the monitoring port number.
Therefore, the embodiment of the application can control implementation difficulty and operation and maintenance difficulty by matching the corresponding monitoring port numbers according to the types of the traffic data, and can flexibly monitor the traffic data.
With reference to the first aspect, in an implementation manner of the present application, before the log monitoring module obtains the target traffic data, the method further includes: and configuring the log monitoring module to generate a format of a monitoring log, wherein the format comprises client IP, time and response status codes.
Therefore, the embodiment of the application can lead the log to be standardized to be output by configuring the format for generating the log, and is convenient for processing the input monitoring log subsequently.
With reference to the first aspect, in an implementation manner of the present application, after the log monitoring module generates a monitoring log based on the target traffic data, the method further includes: and confirming that each virtual machine in the at least one virtual machine generates a corresponding monitoring log, and collecting each monitoring log to a log center for storage.
Therefore, the embodiment of the application can ensure the accuracy of log generation by confirming that each virtual machine generates the corresponding monitoring log.
With reference to the first aspect, in an implementation manner of the present application, before the log monitoring module obtains the target traffic data, the method further includes: downloading a target log monitoring module in a system base image, wherein the target log monitoring module comprises configuration parameters, and the configuration parameters comprise monitoring port numbers; binding the target log monitoring module with the first virtual machine; the log monitoring module obtains target flow data, including: the target log monitoring module acquires the target flow data; the log monitoring module generates a monitoring log based on the target flow data, including: the target log monitoring module generates a monitoring log based on the target flow data.
Therefore, the embodiment of the application can shorten the duration of the configuration parameters of the target equipment and improve the operation efficiency of the target equipment by searching the target log monitoring module with the matched parameters in the basic mirror image.
With reference to the first aspect, in an implementation manner of the present application, after the log monitoring module generates a monitoring log based on the target traffic data, the method further includes: after the first virtual machine is deleted, deleting the log monitoring module bound with the first virtual machine at the same time; or after adding a new first virtual machine, binding the log monitoring module for the new first virtual machine at the same time.
Therefore, according to the embodiment of the application, the log monitoring module can flexibly and dynamically expand the capacity by following the virtual machine, so that the operation load of the target equipment is reduced.
In a second aspect, the present application provides a device for acquiring a monitoring log, where the log monitoring module includes a flow acquisition module and a log generation module; the flow obtaining module is configured to obtain target flow data, and forward the target flow data to a first virtual machine so that the first virtual machine processes the target flow data, wherein the target device comprises at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and the one virtual machine is bound with the one log monitoring module; the log generation module is configured to generate a monitoring log based on the target traffic data.
With reference to the second aspect, in an embodiment of the present application, the flow obtaining module is further configured to: acquiring the request type of the target flow data; searching a monitoring port number corresponding to the request type in a monitoring port comparison table, wherein the monitoring port comparison table is used for storing the corresponding relation between the request type and the monitoring port number; modifying the monitoring port number of the log monitoring module; and the log monitoring module acquires the target flow data through the monitoring port number.
With reference to the second aspect, in an embodiment of the present application, the flow obtaining module is further configured to: and configuring the log monitoring module to generate a format of a monitoring log, wherein the format comprises client IP, time and response status codes.
With reference to the second aspect, in an embodiment of the present application, the log generation module is further configured to: and confirming that each virtual machine in the at least one virtual machine generates a corresponding monitoring log, and collecting each monitoring log to a log center for storage.
With reference to the second aspect, in an embodiment of the present application, the flow obtaining module is further configured to: downloading a target log monitoring module in a system base image, wherein the target log monitoring module comprises configuration parameters, and the configuration parameters comprise monitoring port numbers; binding the target log monitoring module with the first virtual machine; the target log monitoring module acquires the target flow data; the target log monitoring module generates a monitoring log based on the target flow data.
With reference to the second aspect, in an embodiment of the present application, the log generation module is further configured to: after the first virtual machine is deleted, deleting the log monitoring module bound with the first virtual machine at the same time; or after adding a new first virtual machine, binding the log monitoring module for the new first virtual machine at the same time.
In a third aspect, the present application provides a system for obtaining a monitoring log, the system comprising: the client is used for sending the target flow data; and the target device is used for acquiring the target flow data and executing the method according to any embodiment of the first aspect according to the target flow data to acquire a monitoring log.
In a fourth aspect, the present application provides an electronic device, including: a processor, a memory, and a bus; the processor is connected to the memory via the bus, the memory storing a computer program which, when executed by the processor, performs the method according to any embodiment of the first aspect.
In a fifth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed, performs a method according to any embodiment of the first aspect.
Drawings
FIG. 1 is a schematic diagram of a system for obtaining a monitoring log according to an embodiment of the present disclosure;
FIG. 2 is a flowchart of a method for obtaining a monitoring log according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a device for acquiring a monitoring log according to an embodiment of the present application;
fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application based on the embodiments of the present application.
The embodiment of the application can be applied to a scene of generating the monitoring log, and in order to solve the problems in the background technology, in some embodiments of the application, by matching one log monitoring module for each virtual machine, the corresponding monitoring log is generated by the log monitoring module. For example: in some embodiments of the present application, first, a log monitoring module obtains target traffic data and forwards the target traffic data to a first virtual machine, so that the first virtual machine processes the target traffic data, and then the log monitoring module generates a monitoring log based on the target traffic data.
The method steps in the embodiments of the present application are described in detail below with reference to the accompanying drawings.
FIG. 1 provides a block diagram of a system for obtaining a monitoring log in some embodiments of the present application, including a client 110 and a target device 120. Specifically, the client 110 sends target traffic data to the target device 120, and after the target traffic data is acquired, the log monitoring module in the target device 120 forwards the target traffic data to the virtual machine corresponding to the current log monitoring module, and then generates a monitoring log based on the target traffic data.
In the related art, the method for generating the monitoring log includes the following two methods:
one solution is to export the monitoring log at the gateway layer. Traffic sources accepted by a business instance, whether it be a traditional virtual machine service or a micro service, are mainly two: the method comprises the steps of upstream gateway distribution and peer instance inter-calling. Outputting the monitoring log at the gateway layer is the most common one. The scheme has two defects, namely, the gateway instance and the service instance are basically in a one-to-many relationship, and mass service log quantity generated in a large-flow scene is concentrated on a small number of gateway instances, so that IO bottleneck is easy to form. Secondly, the scheme can only monitor the south-north traffic (i.e. the traffic transmitted to the service examples from outside), but cannot monitor the east-west traffic between the service examples (i.e. the traffic accessed between the service examples).
The other scheme is that a ServiceMesh mode in a micro-service scene is adopted, the business flow of a Side-Car process managed business is deployed in a business POD/container, and a log format and an output business monitoring log are customized in the Side-Car process. The scheme is a de facto standard scheme (including log, speed limit, high availability, fusing, access control, etc.) in a micro-service system, but the scheme has higher implementation threshold, is complex to reform and maintain, and is difficult to control by small and medium teams and enterprises. And the scheme is mainly suitable for micro-service scenes, and is not friendly to the traditional virtual machine service architecture.
Based on the defects of the two schemes, the application provides a novel standardized output method of the service monitoring log, an nginx instance (namely a log monitoring module) is installed in the POD/virtual machine where each service instance is located, the business in-out flow of the service instance is taken over, and the access_log of the ginx is taken as a standard monitoring log. The scheme can fully utilize the distributed instance resources of the service under the high-flow scene, is not easy to reach IO bottleneck, and the flow forwarding belongs to stateless operation, and can horizontally expand and contract with the service instance. For example, when the POD (i.e. a space for processing data, similar to the virtual machine) is horizontally expanded, an nginx instance is synchronously deployed in the newly started POD to receive upstream traffic and forward to a service instance (process), and similarly, when the virtual machine is horizontally expanded, an nginx instance is synchronously deployed in the virtual machine to receive upstream traffic and forward to the service instance. The deployment flow based on the nginx is simple, the maintenance threshold is low, the implementation complexity and the maintenance cost are low, and meanwhile, the virtual machine architecture and the micro-service architecture can be compatible.
A method for obtaining a monitoring log performed by a target device in some embodiments of the present application is exemplarily described below. It can be appreciated that the technical scheme of the method for obtaining the monitoring log in the embodiment of the application can be applied to any target device, for example, a server.
To at least solve the problems in the background art, as shown in fig. 2, some embodiments of the present application provide a method for obtaining a monitoring log, where the method includes:
s210, a log monitoring module acquires target flow data and forwards the target flow data to a first virtual machine; s220, the log monitoring module generates a monitoring log based on the target flow data.
It is understood that the first virtual machine includes at least one process, and after the first virtual machine receives the target traffic data, one process of the at least one process processes the target traffic data. For example, the process obtains a file corresponding to the target traffic data.
It should be noted that the target device includes at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and one virtual machine is bound with one log monitoring module.
That is, each virtual machine in the target device corresponds to a log monitoring module, the log monitoring module intercepts target traffic data sent to the virtual machine bound with the log monitoring module, and then forwards the target traffic data to the virtual machine bound with the log monitoring module, and a monitoring log is generated according to the target traffic data. For example, the virtual machine a binds a log monitoring module a, after the log monitoring module a monitors that the client sends the target flow data to the virtual machine a, the log monitoring module a forwards the target flow data to the virtual machine a, the process in the virtual machine a processes the target flow data, and the log monitoring module a generates a monitoring log according to the target flow data.
In one embodiment of the present application, the monitoring port of the log monitoring module needs to be set before S210.
Specifically, first, a request type of target traffic data is acquired. Then, a monitoring port number corresponding to the request type is searched in a monitoring port comparison table, wherein the monitoring port comparison table is used for storing the corresponding relation between the request type and the monitoring port number. And finally, modifying the monitoring port number of the log monitoring module.
That is, the log monitor module needs to explicitly monitor the port before acquiring the target traffic data, and the monitored port number is related to the request type of the target traffic data. The target device confirms the request type of the target traffic data in advance, then searches the monitoring port number according to the request type, and configures the monitoring port number of the log monitoring module. And then acquiring target flow data by the log monitoring module through monitoring the port number.
For example, if the request type is http, then the corresponding snoop port number is http80. If the request type is grpc, the corresponding snoop port number is 80http2.
It should be noted that, while setting the listening port number, a forwarding port corresponding to the request type may also be set.
In one embodiment of the present application, the configuration log monitoring module is further required to generate a format of the monitoring log before S210.
It will be appreciated that the format of the monitoring log includes client IP, local time, response status code, original request line, request processing duration, etc.
That is, in order to be able to generate a standard monitoring log, the present application needs to uniformly configure the format of the monitoring log before acquiring the target flow data. It can be appreciated that the format of the monitoring log can be adjusted according to the production requirement, and the format of the monitoring log is not limited in the application.
In one embodiment of the present application, the log monitoring module configured to be completed may be downloaded before S210. Specifically, first, a target log monitoring module is downloaded in a system base image, wherein the target log monitoring module comprises configuration parameters, and the configuration parameters comprise monitoring port numbers. And binding the target log monitoring module with the first virtual machine, acquiring target flow data by the target log monitoring module, and finally generating a monitoring log by the target log monitoring module based on the target flow data.
That is, for further optimization to reduce the operation duration, the target log monitoring module that has been configured with the log format and/or listening port number may be packaged directly into the system base image, saving download time. The mirror image is used in the virtual machine and directly executed, so that the flow forwarding configuration can be completed.
In one embodiment of the present application, after S220, it is confirmed that each virtual machine in the at least one virtual machine generates a corresponding monitoring log, and each monitoring log is collected into a log center and stored.
In one embodiment of the present application, after S220, the first virtual machine may be scaled down or expanded. Specifically, after the first virtual machine is deleted, the log monitoring module bound with the first virtual machine is deleted at the same time. Or after adding the new first virtual machine, binding the journal monitoring module for the new first virtual machine at the same time.
That is, the log monitoring module can perform capacity reduction and expansion along with the virtual machine to which it is bound. After the first virtual machine is contracted, the corresponding log monitoring module is deleted. After the capacity of the first virtual machine is expanded, the corresponding log monitoring module is correspondingly increased. When the capacity expansion is needed in the traffic peak scene, the steps are executed on the new virtual host, so that the service level expansion of the standard monitoring log can be realized. When shrinking, the virtual machine is removed from SLB (server load balancing) or registry according to the original flow of service.
The method for acquiring the monitoring log by the target device in the present application is described above, and a specific embodiment for acquiring the monitoring log in the present application will be described below.
The method and the device are simultaneously applicable to virtual machine service and micro-service architecture. The implementation difficulty and the operation and maintenance difficulty are effectively controlled while the standardized output of the service monitoring log is realized.
As a specific embodiment of the present application, the implementation process of the virtual machine service architecture is as follows:
the first step: assuming that the monitoring port of the service application is X, and the monitoring port of the rginx (i.e. the log monitoring module) is Y (it can be understood that X and Y are not one port), the virtual machine is initialized in the running environment before the application is started. The nginx is downloaded from the intranet.
And a second step of: the log_format parameter configuration of the nginx is modified (default is/etc/nginx. Conf). The log_format parameter of the ngix determines the log format and content of the ngix log. The customized configuration can be performed according to the own needs, and the fields generally comprise $remote_addr (client IP), $time_local (local time), $request (original request line), $status (response status code), $request_time (request processing duration) and the like.
And a third step of: modifying the configuration of a Server module in an HTTP module of the nginx, configuring a port to be monitored by the nginx through a limit parameter, forwarding a request received by a limit interface to a designated new port by using a proxy_pass parameter in a mode of modifying the location configuration aiming at an upstream HTTP request, so that the port traffic forwarding of the HTTP can be realized, and the forwarding request header configuration is assisted to be modified. If a grpc request is sent upstream, the listen parameter in the Server module needs to be configured in the format of "port number http2", such as "80http2". The proxy_pass is changed to grpc_pass.
Fourth step: observing the nginx log, and after the configuration of the steps is finished, normally accessing the service to generate the nginx log (default path is/var/log/nginx/access. Log). The POD/virtual machine where each service instance is located outputs independent nginx logs, and logs of different instances can be summarized to a log center by combining a log acquisition system.
As another specific embodiment of the present application, the micro service architecture is implemented as follows:
the first step: and making configuration required by the nginx service. The namespace in which the configuration is located (in the configuration example, biz-ngix-test) needs to be consistent with the namespace in which the business instance service is located. The content defined in the default configuration represents the configuration of the nginx instance, whose role is to forward the port request that the nginx listens to the IP and port specified by proxy_pass. And realizing the http traffic of the proxy service instance entrance of the nginx instance. If the entry is grpc traffic, reference is made to the third step of the virtual machine service architecture.
And a second step of: creating/modifying a deployment script of a service, and adding configuration of a nginx container for the original deployment script. Wherein, nalmespace represents the name space where the business is located, and imaging, gap-future, com/gap-future/nginx, and LATEST represents the mirror address of the nginx instance, and volumeMount represents the default configuration content defined by replacing the file specified by the internal parameters of the nginx instance with the configuration in the first step.
And a third step of: the k8s service is created to enable the service to provide service to the outside, and the port parameter specifies an interface (for example, 8080) exposed by the service, and the interface exposed to the outside needs to be consistent with the port monitored by the nginx in the first step, so as to ensure that the traffic is distributed to the ginx, and then the traffic is forwarded to the service instance by the ginx.
In order to solve the problems in the related art, a method that is insensitive to the process and has little influence on the process in daily use is needed to realize standardized output of the monitoring log. Therefore, the method and the device have the advantages that each virtual machine is independently configured with one nginx as the flow agent of the virtual machine, and the standard output log of the nginx is used as the monitoring log of the virtual machine, so that the problem that IO bottleneck is easy to generate due to the fact that burst flow occupies gateway computing resources in a gateway log mode is solved, and meanwhile, the east-west flow access monitoring between the virtual machines is realized. The application uses the computing power resource of the application instance of the virtual machine, can realize the output standard monitoring log on the basis of basically not changing the prior art architecture and not influencing the daily online and operation and maintenance of the virtual machine, and can dynamically expand the capacity along with the virtual machine. The virtual machine is not perceived, and the transformation resistance is small. The method and the device also solve the problems of complex deployment and operation and maintenance of the control plane and the data plane in the micro-service architecture mode. The method and the device are used for solving the output problem of the standard service monitoring log, do not relate to functions such as high availability, speed limiting and fusing, do not need complex control rule configuration and k8s configuration, can complete deployment only by configuring one nginx mirror image, and greatly reduce the deployment threshold. The functions are specific and concise, and daily operation and maintenance treatment is not needed basically.
The specific embodiment of obtaining the monitoring log is described above, and the apparatus for obtaining the monitoring log will be described below.
As shown in fig. 3, some embodiments of the present application provide an apparatus 300 for obtaining a monitoring log, the apparatus comprising: a traffic acquisition module 310 and a log generation module 320.
A flow obtaining module 310, configured to obtain target flow data, and forward the target flow data to a first virtual machine, so that the first virtual machine processes the target flow data, where the target device includes at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and the one virtual machine is bound with one log monitoring module; the log generation module 320 is configured to generate a monitoring log based on the target traffic data.
In one embodiment of the present application, the flow acquisition module 310 is further configured to: acquiring the request type of the target flow data; searching a monitoring port number corresponding to the request type in a monitoring port comparison table, wherein the monitoring port comparison table is used for storing the corresponding relation between the request type and the monitoring port number; modifying the monitoring port number of the log monitoring module; and the log monitoring module acquires the target flow data through the monitoring port number.
In one embodiment of the present application, the flow acquisition module 310 is further configured to: and configuring the log monitoring module to generate a format of a monitoring log, wherein the format comprises client IP, time and response status codes.
In one embodiment of the present application, the log generation module 320 is further configured to: and confirming that each virtual machine in the at least one virtual machine generates a corresponding monitoring log, and collecting each monitoring log to a log center for storage.
In one embodiment of the present application, the flow acquisition module 310 is further configured to: downloading a target log monitoring module in a system base image, wherein the target log monitoring module comprises configuration parameters, and the configuration parameters comprise monitoring port numbers; binding the target log monitoring module with the first virtual machine; the target log monitoring module acquires the target flow data; the target log monitoring module generates a monitoring log based on the target flow data.
In one embodiment of the present application, the log generation module 320 is further configured to: after the first virtual machine is deleted, deleting the log monitoring module bound with the first virtual machine at the same time; or after adding a new first virtual machine, binding the log monitoring module for the new first virtual machine at the same time.
In the embodiment of the present application, the module shown in fig. 3 can implement each process in the embodiments of the methods of fig. 1 and fig. 2. The operation and/or function of the individual modules in fig. 3 are for the purpose of realizing the respective flows in the method embodiments in fig. 1 and 2, respectively. Reference is specifically made to the description in the above method embodiments, and detailed descriptions are omitted here as appropriate to avoid repetition.
As shown in fig. 4, an embodiment of the present application provides an electronic device 400, including: a processor 410, a memory 420 and a bus 430, said processor being connected to said memory by means of said bus, said memory storing computer readable instructions for implementing the method according to any of the above-mentioned embodiments, when said computer readable instructions are executed by said processor, see in particular the description of the above-mentioned method embodiments, and detailed descriptions are omitted here as appropriate for avoiding repetition.
Wherein the bus is used to enable direct connection communication of these components. The processor in the embodiment of the application may be an integrated circuit chip, which has a signal processing capability. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but may also be a Digital Signal Processor (DSP), application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The Memory may be, but is not limited to, random access Memory (Random Access Memory, RAM), read Only Memory (ROM), programmable Read Only Memory (Programmable Read-Only Memory, PROM), erasable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc. The memory has stored therein computer readable instructions which, when executed by the processor, perform the method described in the above embodiments.
It will be appreciated that the configuration shown in fig. 4 is illustrative only and may include more or fewer components than shown in fig. 4 or have a different configuration than shown in fig. 4. The components shown in fig. 4 may be implemented in hardware, software, or a combination thereof.
The embodiments of the present application further provide a computer readable storage medium, on which a computer program is stored, which when executed by a server, implements the method according to any one of the foregoing embodiments, and specifically reference may be made to the description in the foregoing method embodiments, and detailed descriptions are omitted here as appropriate to avoid redundancy.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of obtaining a monitoring log, for application to a target device, the method comprising:
the method comprises the steps that a log monitoring module obtains target flow data and forwards the target flow data to a first virtual machine so that the first virtual machine processes the target flow data, wherein target equipment comprises at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and the virtual machine is bound with the log monitoring module;
the log monitoring module generates a monitoring log based on the target flow data.
2. The method of claim 1, wherein prior to the log monitoring module obtaining target flow data, the method further comprises:
acquiring the request type of the target flow data;
searching a monitoring port number corresponding to the request type in a monitoring port comparison table, wherein the monitoring port comparison table is used for storing the corresponding relation between the request type and the monitoring port number;
modifying the monitoring port number of the log monitoring module;
the log monitoring module obtains target flow data, including:
and the log monitoring module acquires the target flow data through the monitoring port number.
3. The method of claim 1 or 2, wherein prior to the log monitoring module obtaining target traffic data, the method further comprises:
and configuring the log monitoring module to generate a format of a monitoring log, wherein the format comprises client IP, time and response status codes.
4. The method of claim 1 or 2, wherein after the log monitoring module generates a monitoring log based on the target traffic data, the method further comprises:
and confirming that each virtual machine in the at least one virtual machine generates a corresponding monitoring log, and collecting each monitoring log to a log center for storage.
5. The method of claim 1, wherein prior to the log monitoring module obtaining target flow data, the method further comprises:
downloading a target log monitoring module in a system base image, wherein the target log monitoring module comprises configuration parameters, and the configuration parameters comprise monitoring port numbers;
binding the target log monitoring module with the first virtual machine;
the log monitoring module obtains target flow data, including:
the target log monitoring module acquires the target flow data;
the log monitoring module generates a monitoring log based on the target flow data, including:
the target log monitoring module generates a monitoring log based on the target flow data.
6. The method of claim 1 or 2, wherein after the log monitoring module generates a monitoring log based on the target traffic data, the method further comprises:
after the first virtual machine is deleted, deleting the log monitoring module bound with the first virtual machine at the same time; or,
after a new first virtual machine is added, the log monitoring module is bound for the new first virtual machine at the same time.
7. The device for acquiring the monitoring log is characterized by being applied to target equipment, wherein the log monitoring module comprises a flow acquisition module and a log generation module;
the flow obtaining module is configured to obtain target flow data, and forward the target flow data to a first virtual machine so that the first virtual machine processes the target flow data, wherein the target device comprises at least one virtual machine and at least one log monitoring module, the first virtual machine is any one of the at least one virtual machine, and the one virtual machine is bound with the one log monitoring module;
the log generation module is configured to generate a monitoring log based on the target traffic data.
8. A system for obtaining a monitoring log, the system comprising:
the client is used for sending the target flow data;
target device for obtaining the target flow data and for performing the method according to any of claims 1-6 based on the target flow data, obtaining a monitoring log.
9. An electronic device, comprising: a processor, a memory, and a bus;
the processor is connected to the memory via the bus, the memory storing a computer program which, when executed by the processor, performs the method according to any of claims 1-6.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed, implements the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310456533.XA CN116248559A (en) | 2023-04-25 | 2023-04-25 | Method, system, device, equipment and medium for acquiring monitoring log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310456533.XA CN116248559A (en) | 2023-04-25 | 2023-04-25 | Method, system, device, equipment and medium for acquiring monitoring log |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116248559A true CN116248559A (en) | 2023-06-09 |
Family
ID=86631606
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310456533.XA Pending CN116248559A (en) | 2023-04-25 | 2023-04-25 | Method, system, device, equipment and medium for acquiring monitoring log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116248559A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435420A (en) * | 2023-09-12 | 2024-01-23 | 中科驭数（北京）科技有限公司 | DPU-based data plane log acquisition method and system |
-
2023
- 2023-04-25 CN CN202310456533.XA patent/CN116248559A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435420A (en) * | 2023-09-12 | 2024-01-23 | 中科驭数（北京）科技有限公司 | DPU-based data plane log acquisition method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110351283B (en) | Data transmission method, device, equipment and storage medium | |
| CN109474936B (en) | Internet of things communication method and system applied among multiple lora gateways | |
| CN114124451B (en) | Data processing method and system for Internet of things equipment and computer storage medium | |
| CN109040207B (en) | Method, device and equipment for accessing user mode network file system | |
| CN108429739B (en) | Method, system and terminal equipment for identifying honeypots | |
| CN113268308B (en) | Information processing method, device and storage medium | |
| CN112995269B (en) | Data processing method, computer device and readable storage medium | |
| CN116248559A (en) | Method, system, device, equipment and medium for acquiring monitoring log | |
| CN114501593B (en) | Network slice access method, device, system and storage medium | |
| CN113746651B (en) | Method for accessing network slice, electronic equipment and storage medium | |
| CN110597783B (en) | Database management method, device, equipment and storage medium | |
| CN112511366A (en) | Test system, method, device, equipment and storage medium | |
| CN105068926A (en) | Program test method and device thereof | |
| CN111600929B (en) | Transmission line detection method, routing strategy generation method and proxy server | |
| CN112968965A (en) | Metadata service method, server and storage medium for NFV network node | |
| CN106612307B (en) | A kind of implementation method and device of always online business | |
| CN113708869A (en) | Method, device and system for configuring port state | |
| CN110661895A (en) | Network address mapping method and network address mapping equipment of server | |
| CN113422772B (en) | Private network terminal access processing method and device and electronic equipment | |
| CN114827249A (en) | Method and device for extending grid agent | |
| CN115225652A (en) | Method and system for determining edge service platform, electronic equipment and storage medium | |
| CN114500302B (en) | ICE service arrangement method, ICE service arrangement device, terminal equipment and storage medium | |
| CN111478941A (en) | Mock automatic operation method and device, computer equipment and storage medium | |
| CN116979523B (en) | Scene-based substation secondary system simulation method and computer equipment | |
| CN114095502B (en) | Service processing method, system, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |