CN111835562A - Log uploading method, log query method and log query system - Google Patents
Log uploading method, log query method and log query system Download PDFInfo
- Publication number
- CN111835562A CN111835562A CN202010617963.1A CN202010617963A CN111835562A CN 111835562 A CN111835562 A CN 111835562A CN 202010617963 A CN202010617963 A CN 202010617963A CN 111835562 A CN111835562 A CN 111835562A
- Authority
- CN
- China
- Prior art keywords
- log
- initial
- uploading
- uploaded
- error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to a log uploading method, a log query system, a computer device and a storage medium. The method is applied to an edge server and comprises the following steps: responding to a log uploading request, and acquiring an initial log generated by terminal equipment, wherein the terminal equipment and an edge server are in the same communication cluster; if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded; and uploading the log to be uploaded to a cloud server. The method can be understood that the initial logs generated by the terminal equipment which is in the same communication cluster with the edge server are obtained through the edge server, the corresponding screening strategies are selected to screen the initial logs according to the types of the initial logs, logs to be uploaded are obtained and uploaded, deletion and filtration of the initial logs are achieved, the number of uploaded logs is reduced, occupation of network bandwidth is reduced, and stability and efficiency of log uploading are improved.
Description
Technical Field
The present application relates to the field of digital information transmission technologies, and in particular, to a log uploading method, a log querying method, and a log uploading system.
Background
With the development of internet technology, logs (logs) play an increasingly important role. The log is used for recording specific information (including prompts, warnings, error information and the like) in the computer program, and the information can be used for analyzing the running process of the computer program.
In an edge computing scenario, when an application service is deployed at an edge end, the application service can continuously process data collected by machine nodes, and then a large amount of logs are generated. However, because the network of the edge computing scenario is usually unstable, when a large amount of log data is uploaded by using the conventional log uploading method, the problem of log loss occurs, and the log uploading stability is poor.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a log uploading method, a log querying method, a system, a computer device, and a storage medium, which can improve log uploading stability.
A log uploading method is applied to an edge server and comprises the following steps:
responding to a log uploading request, and acquiring an initial log generated by terminal equipment, wherein the terminal equipment and the edge server are in the same communication cluster;
if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded;
and uploading the log to be uploaded to a cloud server.
In one embodiment, the type of the initial log comprises a level of the initial log;
if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded, including:
if the initial log comprises a log with the rank of error, acquiring the log above the log with the rank of error;
taking the log with the rank of error and the log above the log as the log to be uploaded;
and/or the presence of a gas in the gas,
if the initial log comprises the log with the level of debug, discarding the log with the level of debug to obtain the log to be uploaded;
and/or the presence of a gas in the gas,
and if the initial log comprises the log with the level of upload, taking the log with the level of upload as the log to be uploaded.
In one embodiment, if the initial log includes a log of an error level, acquiring an above log of the error level includes:
if the initial log comprises a log with the rank of error, acquiring information of terminal equipment related to the log with the rank of error, wherein the log with the rank of error is generated by the terminal equipment;
and acquiring the upper log of the log with the rank of error according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file, wherein the log file comprises the log with the rank of error and the upper log thereof.
In one embodiment, the obtaining the log of the log with the rank of error according to the information of the terminal device and the corresponding relationship between the terminal device and the log file includes:
and if the log with the error level is the log after rotation, acquiring the log with the error level from the log file of rotation according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file.
In one embodiment, the obtaining an initial log generated by a terminal device in response to a log upload request includes:
responding to a log uploading request, and storing the received initial log sent by the terminal equipment in a message queue through a production thread of the message queue;
and consuming and obtaining the initial log from the message queue through a consumption thread of the message queue.
In one embodiment, the uploading the log to be uploaded to a cloud server includes:
acquiring network transmission parameters between the edge server and the cloud server;
according to the network transmission parameters and a preset current limiting algorithm, determining the uploading speed of the log to be uploaded after current limiting;
and uploading the log to be uploaded to a cloud server at the uploading speed after the current limitation.
In one embodiment, the network transmission parameter comprises a data transmission speed; the determining the uploading speed of the log to be uploaded after current limiting according to the network transmission parameters and a preset current limiting algorithm comprises the following steps:
determining the adding speed of tokens in a token bucket according to the data transmission speed and a preset token bucket algorithm, and adding tokens into the token bucket according to the adding speed of the tokens;
and when the log uploading request is received, processing the log uploading request according to the number of the added tokens in the token bucket, and determining the uploading rate of the log to be uploaded after current limiting.
In one embodiment, the method further comprises:
receiving a log query request aiming at a target service, which is sent by the cloud server, wherein the log query request carries current time;
acquiring a service list according to the log query request;
determining a terminal equipment identifier corresponding to the target service according to the service list;
and obtaining log data of the target service from the log file with the generation time corresponding to the terminal equipment identifier and the current time interval being the latest, and uploading the log data to the cloud server.
A log query method is applied to a cloud server and comprises the following steps:
sending a log query request aiming at a target service to an edge server, wherein the log query request carries current time;
receiving log data of a target service uploaded by the edge server in response to the log query request, wherein the log data are obtained by the edge server according to the log query request, obtaining a resource service list, determining a terminal device identifier corresponding to the target service according to the resource service list, and obtaining the log data from a log file with the generation time corresponding to the terminal device identifier and the current time interval being the closest; and when the type of the initial log generated by the terminal equipment is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain the log data in the log file.
A log upload system comprising an edge server, the edge server comprising:
the uploading module is used for responding to a log uploading request, acquiring an initial log generated by the terminal device, selecting a log screening strategy corresponding to a screening type to screen the initial log if the type of the initial log is the screening type, obtaining a log to be uploaded, and uploading the log to be uploaded to the cloud server, wherein the terminal device and the edge server are in the same communication cluster.
In one embodiment, the edge server further comprises a storage management module;
the uploading module is further configured to send the received initial log sent by the terminal device to a storage management module of the edge server;
the storage management module is used for storing the initial log sent by the terminal equipment into a message queue through a production thread of the message queue;
the uploading module is further configured to obtain the initial log from the message queue through a consumption thread of the message queue;
the edge server also comprises a log query module and an index management module; the type of the initial log comprises a level of the initial log;
the uploading module is specifically configured to, if the initial log includes a log of which the level is error, obtain, by the log query module, an upper log of the log of which the level is error, and use the log of which the level is error and the upper log thereof as the log to be uploaded;
the log query module is configured to acquire information of the terminal device associated with the log of the rank error, and acquire the log above the log of the rank error according to the information of the terminal device and a correspondence between the terminal device and a log file acquired by the index management module, where the log of the rank error is generated by the terminal device.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
responding to a log uploading request, and acquiring an initial log generated by terminal equipment, wherein the terminal equipment and the edge server are in the same communication cluster;
if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded;
and uploading the log to be uploaded to a cloud server.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
responding to a log uploading request, and acquiring an initial log generated by terminal equipment, wherein the terminal equipment and the edge server are in the same communication cluster;
if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded;
and uploading the log to be uploaded to a cloud server.
According to the log uploading method, the log query system, the computer equipment and the storage medium, in response to a log uploading request, an initial log generated by terminal equipment in the same communication cluster with an edge server is obtained through the edge server, and then when the type of the initial log is a screening type, a corresponding log screening strategy is selected to screen the initial log, so that a log to be uploaded is obtained and uploaded. According to the method and the device, the original logs are deleted and filtered, the number of uploaded logs is reduced, the occupation of network bandwidth is reduced, and the log uploading stability and efficiency are improved.
Drawings
FIG. 1 is a diagram of an application environment for a log upload method in one embodiment;
FIG. 2 is a flowchart illustrating a log uploading method according to an embodiment;
fig. 3 is a schematic flowchart illustrating a supplementary scheme for selecting a log screening policy corresponding to a screening type to screen an initial log if the level type of the initial log is the screening type in one embodiment;
FIG. 4 is a flow diagram that illustrates asynchronous transmission of logs using message queues, according to one embodiment;
FIG. 5 is a schematic flow diagram illustrating real-time query logging, according to one embodiment;
FIG. 6 is a flow diagram that illustrates a method for log queries in one embodiment;
FIG. 7 is a diagram of an application framework for a log upload method in one embodiment;
FIG. 8 is a block diagram of an edge server in one embodiment;
FIG. 9 is a block diagram of an apparatus for log upload in one embodiment;
FIG. 10 is a block diagram showing the structure of a log search device according to an embodiment;
FIG. 11 is a diagram illustrating an internal structure of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The log uploading method and the log query method provided by the application can be applied to the application environment shown in fig. 1. The terminal device 102 communicates with the edge server 104 through a network, and the edge server 104 communicates with the cloud server 106 through the network. The log data generated by the terminal device 102 is sent to the edge server 104, the edge server 104 collects the log data to obtain an initial log, and screens the initial log according to the type of the initial log to obtain a log to be uploaded and uploads the log to the cloud server 106. The terminal device 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the edge server 104 and the cloud server 106 may be implemented by independent servers or a server cluster formed by a plurality of servers.
In an exemplary embodiment, as shown in fig. 2, a log uploading method is provided, which is described by taking the method as an example applied to the edge server in fig. 1, and includes the following steps:
step S202, responding to the log uploading request, and acquiring an initial log generated by the terminal equipment.
The terminal equipment and the edge server are in the same communication cluster. In one embodiment, the Kubernets engine is embedded in the edge server and the terminal equipment, so that the construction of a communication cluster of the edge server and the terminal equipment is realized.
In one embodiment, the log upload request may be an upload request sent by the terminal device to the edge server for log data generated by the terminal device. The edge server receives the log uploading request and responds to the terminal equipment to obtain an initial log generated by the terminal equipment. In another embodiment, the log uploading request may be an uploading request automatically generated by the edge server after receiving the log data sent by the terminal device. In this case, the edge server, in response to the log upload request, first stores the initial log in the memory of the edge server, and then reads or consumes the initial log from the memory.
In one embodiment, the initial log may be a log generated by each end device managed by a kubernets engine. For example, the initial log may be a log generated by each terminal device and sent by the terminal device to the edge server. Further, a log collector may be embedded in a kubernets engine in the terminal device, and configured to collect logs generated by each terminal device, and send the collected logs to the edge server through the log collector. By adopting the log collector, the log data generated by each terminal device can be managed in a centralized manner, the frequency of sending the logs to the edge server is reduced, and the IO pressure of the edge server is reduced.
And step S204, if the type of the initial log is the screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain the log to be uploaded.
In the log processing and transmission process, the logs are classified in order to achieve efficient and accurate processing and transmission due to the large volume of the logs. Different log classification strategies are adopted according to different requirements. In this step, a filtering type is set for filtering the initial log. The screening type is used for identifying and screening the log of the type from a plurality of log data. And aiming at different screening types, different screening strategies can be selected, so that the aim of diversified screening is fulfilled.
Specifically, after determining the type of the initial log, the edge server judges whether the type is a preset screening type, and if so, selects a log screening strategy corresponding to the screening type to screen the initial log, so as to obtain the log to be uploaded. The log screening strategy comprises deleting the log, obtaining the log above the log, directly uploading the log without processing, and the like.
And step S206, uploading the log to be uploaded to a cloud server.
Specifically, the edge server uploads the log to be uploaded to the cloud server. Optionally, the edge server uploads the log to be uploaded to the cloud server through an HTTP interface exposed by the cloud server.
In the log uploading method, in response to a log uploading request, an edge server acquires an initial log generated by a terminal device in the same communication cluster as the edge server, and selects a corresponding log screening strategy to screen the initial log when the type of the initial log is a screening type, so as to obtain a log to be uploaded and upload the log. The method can be understood to delete and filter the initial log, reduce the amount of uploaded logs, reduce the occupation of network bandwidth and improve the stability and efficiency of log uploading.
To improve the performance of kubernets engines in communication clusters, in an exemplary embodiment, the edge server operates in a Deployment fashion through the kubernets engine. In the embodiment of the application, the edge server operates in a Deployment mode, so that only one edge server is deployed on an edge end cluster (a communication cluster formed by the edge server and the terminal device) to provide services, and the services are not limited to operate on a certain terminal device node and only need to operate on the edge end cluster.
In an exemplary embodiment, the present invention relates to a possible implementation manner of obtaining an initial log generated by a terminal device. On the basis of the above embodiment, step S202 may be specifically implemented by the following steps:
step S2022, receiving the initial log sent by the log collector.
The initial log is obtained by collecting log data sent by each terminal device received by a log collector. The Kubernetes engine is embedded in the terminal equipment. The log collector is embedded in a Kubernetes engine in the terminal equipment.
Specifically, first, a kubernets engine is deployed in the terminal device, so that the terminal device and the edge server together form an edge terminal with respect to a cloud where the cloud server is located. A cluster is built at an edge end through a Kubernetes engine, a log collector is arranged at each terminal equipment node in the Kubernetes engine, log data generated by each terminal equipment node can be collected through the log collector, the collected log data are collected to obtain an initial log, and the initial log is sent to an edge server, so that the edge server can receive the initial log, and screening, storage and uploading of subsequent initial logs are realized.
In the embodiment of the application, the Kubernetess engine is adopted to manage the edge end cluster, so that deployment implementation and high availability can be realized more conveniently.
In an exemplary embodiment, the log collector comprises a Fluent-bit collector. The Fluent-bit collector is embedded in a Kubernetes engine in the terminal equipment and runs in a DaemonSet mode, wherein the DaemonSet mode can enable each terminal equipment node on the Kubernetes cluster to run an application service Pod. Specifically, the Fluent-bit collector acquires logs managed by a Kubernets engine under the conditions of/var/log/contacts in a tail command mode, wherein the/var/log/contacts is a file path of the logs stored on a terminal equipment node, and the path is uniformly managed by the Kubernets engine. And the Fluent-bit collector uses the kubernets filters to filter and format package the acquired logs to obtain initial logs. The format encapsulation encapsulates information related to the kubernets engine, for example, after encapsulating a log, the log may contain tag data such as k8s _ container _ name, k8s _ pod, k8s _ node _ name, and k8s _ pod _ name. And then, the Fluent-bit collector sends the packaged initial log to an edge server through an http output interface. The edge server may receive the initial log through an HTTP interface.
In the embodiment of the application, the Fluent-bit collector is adopted to collect the log data of each terminal equipment node, so that the resource consumption is low, the strength of using resources is lighter, and the method is more suitable for calculation of the edge end.
In another exemplary embodiment, the journal logger comprises a fluent logger. The fluent collector is embedded in a Kubernetes engine in the terminal equipment. In the embodiment of the application, the fluent collector is adopted to collect the log data of each terminal equipment node, and the stability is higher.
In an exemplary embodiment, the type of initial log includes a level of the initial log. Wherein, the level of the initial log can be set according to the actual function requirement. In one embodiment, the level of the initial log includes one or more of error, info, debug, war, or upload. In one embodiment, the level of the initial log may be determined based on the particular fields contained in the initial log. Alternatively, the specific field may be a level name, for example, an error level log contains an "error" field, an info level log contains an "info" field, and so on. Based on this, in an exemplary embodiment, the log screening method relates to a possible implementation manner that if the type of the initial log is the screening type, a log screening policy corresponding to the screening type is selected to screen the initial log to obtain the log to be uploaded. On the basis of the above embodiment, step S204 can be specifically implemented by the following steps:
step S212, if the initial log comprises the log with the rank of error, acquiring the log above the log with the rank of error;
step S214, the log with rank error and the log above it are used as the log to be uploaded.
Wherein error indicates that a system error and an exception occur, and the target operation cannot be completed normally.
Specifically, for a log of level error, the edge server obtains the above log of the log of level error. Wherein, the above log refers to the above content of the log. In one embodiment, assuming that a log with a certain level of error is located in the 100 th row of the log file, the upper log is all logs of the 1 st row to 99 th row in the log file. In another embodiment, the size of the above log may also be configured in advance, for example, the above log of the log with the rank of error is 20 lines, and the above log is the log of the first 20 lines of the log with the rank of error. And then, the edge server takes the log with the error level and the upper log thereof as the log to be uploaded, and uploads the log to the cloud server.
Optionally, if the initial log includes a log of an error field, the log is determined as a log of a level of error. Of course, other specific fields may be set as the identifier for the log with the identification level of error, such as the error field.
In the embodiment of the application, the logs with the rank of error are adopted, and under the condition that the logs with the rank of error record error content, the logs matched with the logs with the rank of error are uploaded to the cloud server together, so that the log back-check function is realized, and the positions of errors and abnormalities of the system can be accurately positioned.
In an exemplary embodiment, the method involves obtaining the above log of the log with the error level if the initial log includes the log with the error level. On the basis of the above embodiment, step S212 may be specifically implemented by the following steps:
step S2122, if the initial log comprises the log with the rank of error, acquiring information of the terminal equipment related to the log with the rank of error;
step S2124, acquiring the log with the rank of error according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file.
Wherein, the log with the rank of error is generated by the terminal equipment associated with the log. The log file comprises logs with the rank of error and the logs above.
Specifically, if the initial log includes a log of which the level is error, the edge server queries information of a terminal device associated with the log of which the level is error, such as a terminal device ID, establishes a communication connection with the terminal device through an ssh network protocol, and queries an upper log of the log of which the level is error according to a correspondence between the terminal device and a log file under/var/log/contacts.
In an exemplary embodiment, the present invention relates to a possible implementation manner of obtaining the above log of the log with the error level according to the information of the terminal device and the corresponding relationship between the terminal device and the log file. On the basis of the foregoing embodiment, step S2122 may be specifically implemented by the following steps:
in step S212a, if the log with the rank error is the log after rotation, the log above the log with the rank error is obtained from the log file of rotation according to the information of the terminal device and the corresponding relationship between the terminal device and the log file.
Wherein, the log rotation is used for controlling the size of the log file. And if the data volume of the log in a certain log file reaches a preset threshold value, performing one-time rotation on the log so as to write a new log into a new log file.
Specifically, if the log with the rank of error is the log after rotation, the edge server acquires the above log of the log with the rank of error from the log file of rotation according to the information of the terminal device and the corresponding relationship between the terminal device and the log file. Optionally, the edge server obtains the position of the log file of the rotation where the log of the level error is located according to the information of the terminal device and the hierarchical mapping relationship of the Namespace, the Node, the Pod, and the Container, and queries and obtains the above log of the level error from the log file of the rotation.
In the embodiment of the application, after the log to be uploaded is rotated, the upper log can still be accurately found, and the log query accuracy is improved. And further, the log query is realized by using the hierarchical mapping relation of Namespace, Node, Pod and Container as an index, which is beneficial to improving the query use efficiency when the edge server uploads the log.
In an exemplary embodiment, if the type of the initial log is the screening type, a log screening policy corresponding to the screening type is selected to screen the initial log, so as to obtain another possible implementation manner of the log to be uploaded. On the basis of the above embodiment, step S204 can be specifically implemented by the following steps:
in step S222, if the initial log includes the log with the level of debug, the log with the level of debug is discarded, and the log to be uploaded is obtained.
The debug level is the lowest level, and the debug level is used for knowing the running state of the system in more detail during debugging.
Specifically, if the initial log includes a log with the level of debug, the edge server discards the log with the level of debug to obtain a log to be uploaded. For the logs with the debug level, because a large number of logs with the debug level are generated in the scene of edge computing, but the logs with the debug level are not required to be uploaded to the cloud server, the logs with the debug level need to be discarded, deletion and filtering of the initial logs are achieved, the transmission quantity of the logs is reduced, occupation of network bandwidth is reduced, and the log uploading efficiency is improved.
Optionally, if the log of the debug field is included in the initial log, the log is determined to be a log of a debug level. Of course, other specific fields, such as the deb field, may be set as the identifier for identifying the log with the debug level.
In the embodiment of the application, logs of which the uploading level is not required are abandoned, deletion and filtration of the initial logs are achieved, and the uploading efficiency of the logs can be effectively improved.
In an exemplary embodiment, the present invention relates to another possible implementation manner in which, if the type of the initial log is a screening type, a log screening policy corresponding to the screening type is selected to screen the initial log to obtain a log to be uploaded. On the basis of the above embodiment, step S204 can be specifically implemented by the following steps:
step S232, if the initial log includes the log with the level of upload, the log with the level of upload is used as the log to be uploaded.
Specifically, if the initial log includes a log of which the level is upload, the edge server does not process the log of which the level is upload, directly takes the log of which the level is upload as a log to be uploaded, and uploads the log to the cloud server.
Optionally, if the initial log includes a log of the upload field, the log is determined to be a log of the upload level. Of course, other specific fields may be set as the identification of the log with the identification level of upload, such as the upl field.
In the embodiment of the application, the log with the upload level, which needs to be uploaded, is uploaded to the cloud server, so that operation and maintenance personnel can call the log at any time and can know the running state of the system in time.
In an exemplary embodiment, step S202 may be specifically implemented by the following steps:
step S242, in response to the log upload request, storing the received initial log sent by the terminal device in the message queue through the production thread of the message queue;
in step S244, an initial log is obtained from the message queue by consuming through the consuming thread of the message queue.
The message queue is provided with a production thread and a consumption thread, so that asynchronous transmission of logs can be realized.
Specifically, when the edge server receives a log uploading request automatically generated from log data sent by the terminal device, the edge server responds to the log uploading request, and first stores an initial log, specifically, the edge server stores the initial log in a message queue through a production thread of the message queue. Optionally, if the stored data amount reaches the data amount threshold, the stored history log is deleted. The high availability of storage is improved by regularly arranging the historical log data. And then, the edge server consumes the initial log from the message queue through the consumption thread of the message queue. Optionally, the consumption thread reads logs with a preset size according to the configuration information, and merges the logs with the preset size into a network message to be uploaded to the cloud server, so that the network load pressure can be reduced.
In the embodiment of the application, the asynchronous uploading of the log is realized by adopting the message queue, especially when the network condition is not good or the network bandwidth is insufficient, the asynchronous uploading can reduce the waiting time generated when the log writing speed is greater than the uploading speed, the stability and the reliability of the edge calculation can be improved, and the high availability effect of the edge calculation can be realized by balancing the storage and the network resources.
In an exemplary embodiment, step S206 may be specifically implemented by the following steps:
step S2062, acquiring network transmission parameters between the edge server and the cloud server;
step S2064, determining the uploading speed of the log to be uploaded after current limiting according to the network transmission parameters and a preset current limiting algorithm;
and S2066, uploading the log to be uploaded to the cloud server at the uploading speed after current limiting.
Wherein the network transmission parameter includes at least one of a data transmission speed, a network bandwidth, a timeout retransmission time, or a round trip delay. The preset current limiting algorithm includes at least one of a token bucket algorithm or a leaky bucket algorithm.
Specifically, the edge server firstly obtains network transmission parameters between the edge server and the cloud server, then determines the uploading speed of the log to be uploaded after current limiting according to the network transmission parameters and a preset current limiting algorithm, and further uploads the log to be uploaded to the cloud server at the uploading speed after current limiting.
Optionally, in an embodiment, step S2064 may be specifically implemented by the following steps:
step S206a, determining the adding speed of tokens in the token bucket according to the data transmission speed and a preset token bucket algorithm, and adding tokens into the token bucket according to the adding speed of the tokens;
step S206b, when receiving the log upload request, processing the log upload request according to the number of the added tokens in the token bucket, and determining the upload rate of the log to be uploaded after current limiting.
Specifically, when the edge server uploads the log to be uploaded, the token bucket algorithm can be adopted to limit the uploading rate of the log to be uploaded, so that the influence of too high network occupation on other edge computing services is prevented. Further, the edge server performs network fault tolerance processing when uploading the log, so that the high availability effect of network processing is realized.
To enable real-time querying of the log, in an exemplary embodiment, the method further comprises the steps of:
step S252, receiving a log query request aiming at a target service, which is sent by a cloud server;
step S254, according to the log query request, obtaining a service list;
step S256, determining a terminal equipment identifier corresponding to the target service according to the service list;
step S258, obtaining log data of the target service from the log file with the generation time corresponding to the terminal device identifier and the current time interval being the latest, and uploading the log data to the cloud server.
The log query request carries the current time.
Specifically, the edge server receives a log query request for a target service sent by the cloud server, acquires a service list capable of being queried, and determines a terminal device identifier corresponding to the target service according to the service list, so that a log file closest to a current time interval is queried in the terminal device, log data of the target service contained in the log file is acquired, and the log data is uploaded to the cloud server. In one embodiment, the edge server determines the terminal device identifier corresponding to the target service according to the hierarchical mapping relationship among the Namespace, the Node, the Pod, and the Container, and queries the log file closest to the current time interval in the terminal device through ssh tail/var/log/contacts.
In an exemplary embodiment, as shown in fig. 6, a log query method is provided, which is described by taking the method as an example applied to the cloud server in fig. 1, and includes the following steps:
step S302, a log query request aiming at the target service is sent to the edge server.
Step S304, receiving the log data of the target service uploaded by the edge server in response to the log query request.
Wherein the log query request carries the current time. The log data is obtained by the edge server through obtaining a resource service list according to the log query request, determining a terminal device identifier corresponding to a target service according to the resource service list, and obtaining the log data from a log file with the generation time corresponding to the terminal device identifier and the current time interval being the closest, wherein the log data in the log file is obtained through selecting a log screening strategy corresponding to a screening type to screen an initial log when the type of the initial log generated by the terminal device is the screening type.
Specifically, the cloud server sends a log query request for the target service to the edge server. The method comprises the steps that an edge server receives a log query request aiming at a target service sent by a cloud server, obtains a service list capable of being queried, and determines a terminal device identifier corresponding to the target service according to the service list, so that a log file closest to the current time interval is queried in the terminal device, log data of the target service contained in the log file are obtained, and the log data are uploaded to the cloud server. In one embodiment, the edge server determines the terminal device identifier corresponding to the target service according to the hierarchical mapping relationship among the Namespace, the Node, the Pod, and the Container, and queries the log file closest to the current time interval in the terminal device through sshtail/var/log/contacts.
According to the log query method, the log query request is sent to the edge server, so that the edge server queries and uploads the latest log data of the target service according to the configuration information, and the function that the cloud server queries the log from the edge server in real time is achieved.
An embodiment of the present disclosure is described below with reference to a specific application scenario, and is specifically illustrated in fig. 7 and 8.
In an exemplary embodiment, there is provided a log uploading system including an edge server including:
the uploading module is used for responding to the log uploading request, obtaining an initial log generated by the terminal device, selecting a log screening strategy corresponding to the screening type to screen the initial log if the type of the initial log is the screening type, obtaining a log to be uploaded, and uploading the log to be uploaded to the cloud server, wherein the terminal device and the edge server are in the same communication cluster.
Further, in an exemplary embodiment, the edge server further comprises a storage management module; the uploading module is also used for sending the received initial log sent by the terminal equipment to a storage management module of the edge server;
the storage management module is used for storing the initial log sent by the terminal equipment into the message queue through a production thread of the message queue;
the uploading module is also used for consuming and obtaining an initial log from the message queue through a consuming thread of the message queue, wherein the type of the initial log comprises the level of the initial log;
the edge server also comprises a log query module and an index management module;
the uploading module is specifically used for acquiring the log with the error level through the log query module if the initial log contains the log with the error level, and taking the log with the error level and the log above the log as logs to be uploaded;
and the log query module is used for acquiring the information of the terminal equipment related to the log with the rank of error, and acquiring the log above the log with the rank of error according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file acquired by the index management module, wherein the log with the rank of error is generated by the terminal equipment.
Specifically, as shown in fig. 7 and 8, on the edge end cluster side:
after the Fluent-bit module is deployed successfully, logs generated by each terminal equipment node (including application programs generated by the logs) are collected. The Fluent-bit module is configured to obtain log data managed by Kubernets under the conditions of var/log/contacts in a tail mode, packages relevant information of the Kubernets on the collected initial logs by using a kubernets filter, and sends the collected initial logs to an HTTP interface of the edge server through configured HTTP output.
The edge server deploys and runs a Kubernetes engine, and receives the initial log sent by the Fluent-bit module through the HTTP interface. The edge server comprises the following modules:
and the uploading module (Uploader) receives the initial log through the HTTP interface, sends the initial log to the storage management module for storage, and starts the multithreading call storage management module to consume the initial log and upload the initial log to the cloud. If the consumed initial log is matched with a preset screening condition, for example, the initial log comprises a log with the rank of error, the log query management module is used for acquiring the log above the log according to the consumption state of the log with the rank of error, and the log above the log are uploaded to the cloud server. Optionally, when the log to be uploaded is uploaded to the cloud server by the uploading module, the log to be uploaded is uploaded asynchronously and in batches. Optionally, the network occupation is prevented from being too high to affect other edge computing services by using a TokenBucket algorithm for current limiting. Optionally, the log is subjected to network fault tolerance processing during uploading, so that high availability of network processing is realized.
And the Storage-manager module (Storage-manager) constructs a local message queue and temporary Storage, and is used for reading and writing the stored logs and providing a reading and writing interface. And meanwhile, the storage management is supported, the capacity upper limit is configured, and if the usage amount of the storage reaches the upper limit or reaches the configuration, the historical data is regularly sorted. Considering that the edge network may be poor, the speed of writing to the upload module may be greater than the speed of consuming uploading by the upload module to the cloud server, and therefore, the problem needs to be solved through an asynchronous message queue.
And the Log query management module (Log-query-manager) is used for performing Log query on the Log file. When the uploading module acquires the above log of a certain log by using the log query management module, the uploading module is connected to the terminal equipment node through ssh according to the information of the terminal equipment node of the log, queries/var/log/contacts logs, and simultaneously supports the processing of log rotation configured by Kubernetes, if the log is processed by rotation, the log query management module calls the index management module to acquire the position of a log file of the rotation, and queries the content of the log file of the rotation to acquire the above log.
An Index management module (Index-manager) for providing a Kubernets cluster-level resource relationship list. The index management module acquires mapping relations (maps) of Namespace, Node, Pod and Container by calling Kubernets API Server, and generates cache data for other modules of the edge Server or HTTP interfaces.
Interface modules (APIs): the edge server includes three interfaces: log-receive interface: receiving a log data interface sent by a fluent-bitHTTP output; a/cluster-query-list interface: providing a service interface for the cloud, and acquiring a resource service list which can be queried in real time at present through the interface; (iii) a/cluster-query-log interface: a service interface provided to the cloud server, through which the service is specified, and a log of the service (Pod/Container) is queried.
At the cloud cluster side:
and the Cloud gateway (Cloud gateway) receives the log uploaded by the edge server through the HTTP interface. The cloud gateway is used for constructing the cloud gateway for better expansibility and stability when the number of edge computing devices is large so as to support better services such as traffic management, monitoring and degradation. The cloud gateway directly asynchronously transcribes the logs to Kafka. The edge server can be responded to more quickly in an asynchronous mode, and the network on the edge side is prevented from being blocked.
Kafka, for reducing the load of the logging storage service elasticsearch (es) at high concurrency.
Kafka Consumer writes Kafka's log data to the ElasticSearch, while configuring the ES index based on the log, currently configuring the ES index by service, day.
Elastic search: the method is used for storing log data, constructing ES indexes and facilitating the query of historical log data.
Kibana: after Kibana configures ES, the logs can be screened by selecting ES indexes and Kubernetesfilter of the logs.
When the Cloud server queries the log in real time, the Cloud WEBUI selects the edge end cluster and queries an exposed interface of the edge server deployed in the edge end cluster. The service of the edge server on Kubernetes is inquired through VPN IP and KubernetesNodePort. Specifically, firstly, a service list capable of being queried is obtained through a/group-query-list interface. And then, calling an index management module to acquire the mapping relation of the whole resource and returning the mapping relation to the cloud server as a response. And then inquiring a real-time log of a certain service through a/closed-query-log interface, after receiving the request, the edge server inquires the log of the service through a log inquiry management module, the log inquiry management module acquires a terminal equipment node where the service is located through all resource relations provided by an index management module, and acquires the latest log data in a mode of ssh tail/var/log/contacts/log files.
It should be understood that although the various steps in the flow charts of fig. 2-6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-6 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In an exemplary embodiment, as shown in fig. 9, there is provided a log uploading apparatus for an edge server, including: a log obtaining module 402, a log screening module 404, and a log uploading module 406, wherein:
the log obtaining module 402 is configured to, in response to a log uploading request, obtain an initial log generated by a terminal device, where the terminal device and an edge server are in the same communication cluster.
The log screening module 404 is configured to select a log screening policy corresponding to the screening type to screen the initial log if the type of the initial log is the screening type, so as to obtain a log to be uploaded.
The log uploading module 406 is configured to upload a log to be uploaded to a cloud server.
In the log uploading device, in response to a log uploading request, an edge server acquires an initial log generated by a terminal device in the same communication cluster as the edge server, and selects a corresponding log screening strategy to screen the initial log when the type of the initial log is a screening type, so as to obtain a log to be uploaded and upload the log. It can be understood that the device realizes the deletion and filtration of the initial log, reduces the uploading amount of the log, reduces the occupation of network bandwidth, and improves the uploading stability and efficiency of the log.
In an exemplary embodiment, the log screening module 404 is specifically configured to, if the initial log includes a log with a rank of error, obtain an above log of the log with the rank of error; taking the logs with the error level and the logs above the logs as logs to be uploaded; and/or if the initial log comprises the log with the debug level, discarding the log with the debug level to obtain the log to be uploaded; and/or if the initial log comprises the log with the level of upload, taking the log with the level of upload as the log to be uploaded.
In an exemplary embodiment, the log screening module 404 is specifically configured to, if the initial log includes a log with a level of error, obtain information of a terminal device associated with the log with the level of error, where the log with the level of error is generated by the terminal device; and acquiring the upper log of the log with the rank of error according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file, wherein the log file comprises the log with the rank of error and the upper log thereof.
In an exemplary embodiment, the log screening module 404 is specifically configured to, if the log with the rank of error is a log after rotation, obtain the above log of the log with the rank of error from a log file of rotation according to the information of the terminal device and the corresponding relationship between the terminal device and the log file.
In an exemplary embodiment, the log obtaining module 402 is specifically configured to, in response to a log uploading request, store a received initial log sent by a terminal device in a message queue through a production thread of the message queue; and consuming the initial log from the message queue through a consuming thread of the message queue.
In an exemplary embodiment, the log uploading module 406 is specifically configured to obtain a network transmission parameter between the edge server and the cloud server; according to the network transmission parameters and a preset current limiting algorithm, determining the uploading speed of the log to be uploaded after current limiting; and uploading the log to be uploaded to the cloud server at the uploading speed after the current limitation.
In an exemplary embodiment, the log uploading module 406 is specifically configured to determine an adding speed of tokens in the token bucket according to a data transmission speed and a preset token bucket algorithm, and add tokens to the token bucket according to the adding speed of tokens; and when receiving the log uploading request, processing the log uploading request according to the number of the added tokens in the token bucket, and determining the uploading rate of the log to be uploaded after current limiting.
For specific limitations of the log uploading device, reference may be made to the above limitations of the log uploading method, which is not described herein again. All or part of each module in the log uploading device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In an exemplary embodiment, as shown in fig. 10, there is provided a log query apparatus for a cloud server, including: a request sending module 502 and a log receiving module 504, wherein:
the request sending module 502 is configured to send a log query request for a target service to an edge server, where the log query request carries a current time.
The log receiving module 504 is configured to receive log data of a target service uploaded by an edge server in response to a log query request, where the log data is obtained by the edge server according to the log query request, obtain a resource service list, determine a terminal device identifier corresponding to the target service according to the resource service list, and obtain the log data from a log file whose generation time corresponding to the terminal device identifier is closest to a current time interval; and when the log data in the log file is of a screening type according to the type of the initial log generated by the terminal equipment, selecting a log screening strategy corresponding to the screening type to screen the initial log.
In the log query device, the edge server queries and uploads the latest log data of the target service according to the configuration information by sending the log query request to the edge server, so that the function of querying the log from the edge server in real time by the cloud server is realized.
For the specific limitation of the log query device, reference may be made to the above limitation on the log query method, which is not described herein again. The modules in the log query device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 11. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a log upload method.
Those skilled in the art will appreciate that the architecture shown in fig. 11 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A log uploading method is applied to an edge server and comprises the following steps:
responding to a log uploading request, and acquiring an initial log generated by terminal equipment, wherein the terminal equipment and the edge server are in the same communication cluster;
if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded;
and uploading the log to be uploaded to a cloud server.
2. The method of claim 1, wherein the type of the initial log comprises a level of the initial log;
if the type of the initial log is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain a log to be uploaded, including:
if the initial log comprises a log with the rank of error, acquiring the log above the log with the rank of error;
taking the log with the rank of error and the log above the log as the log to be uploaded;
and/or the presence of a gas in the gas,
if the initial log comprises the log with the level of debug, discarding the log with the level of debug to obtain the log to be uploaded;
and/or the presence of a gas in the gas,
and if the initial log comprises the log with the level of upload, taking the log with the level of upload as the log to be uploaded.
3. The method according to claim 2, wherein if the initial log includes a log of an error level, acquiring a previous log of the error level, including:
if the initial log comprises a log with the rank of error, acquiring information of terminal equipment related to the log with the rank of error, wherein the log with the rank of error is generated by the terminal equipment;
and acquiring the upper log of the log with the rank of error according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file, wherein the log file comprises the log with the rank of error and the upper log thereof.
4. The method according to claim 3, wherein the obtaining the above log of the log with the rank of error according to the information of the terminal device and the corresponding relationship between the terminal device and the log file comprises:
and if the log with the error level is the log after rotation, acquiring the log with the error level from the log file of rotation according to the information of the terminal equipment and the corresponding relation between the terminal equipment and the log file.
5. The method of claim 1, wherein the obtaining of the initial log generated by the terminal device in response to the log uploading request comprises:
responding to a log uploading request, and storing the received initial log sent by the terminal equipment in a message queue through a production thread of the message queue;
and consuming and obtaining the initial log from the message queue through a consumption thread of the message queue.
6. The method of claim 1, wherein uploading the log to be uploaded to a cloud server comprises:
acquiring network transmission parameters between the edge server and the cloud server;
according to the network transmission parameters and a preset current limiting algorithm, determining the uploading speed of the log to be uploaded after current limiting;
and uploading the log to be uploaded to a cloud server at the uploading speed after the current limitation.
7. The method of claim 6, wherein the network transmission parameter comprises a data transmission speed; the determining the uploading speed of the log to be uploaded after current limiting according to the network transmission parameters and a preset current limiting algorithm comprises the following steps:
determining the adding speed of tokens in a token bucket according to the data transmission speed and a preset token bucket algorithm, and adding tokens into the token bucket according to the adding speed of the tokens;
and when the log uploading request is received, processing the log uploading request according to the number of the added tokens in the token bucket, and determining the uploading rate of the log to be uploaded after current limiting.
8. The method of claim 1, further comprising:
receiving a log query request aiming at a target service, which is sent by the cloud server, wherein the log query request carries current time;
acquiring a service list according to the log query request;
determining a terminal equipment identifier corresponding to the target service according to the service list;
and obtaining log data of the target service from the log file with the generation time corresponding to the terminal equipment identifier and the current time interval being the latest, and uploading the log data to the cloud server.
9. A log query method is applied to a cloud server and comprises the following steps:
sending a log query request aiming at a target service to an edge server, wherein the log query request carries current time;
receiving log data of a target service uploaded by the edge server in response to the log query request, wherein the log data are obtained by the edge server according to the log query request, obtaining a resource service list, determining a terminal device identifier corresponding to the target service according to the resource service list, and obtaining the log data from a log file with the generation time corresponding to the terminal device identifier and the current time interval being the closest; and when the type of the initial log generated by the terminal equipment is a screening type, selecting a log screening strategy corresponding to the screening type to screen the initial log to obtain the log data in the log file.
10. A log upload system comprising an edge server, the edge server comprising:
the uploading module is used for responding to a log uploading request, acquiring an initial log generated by the terminal device, selecting a log screening strategy corresponding to a screening type to screen the initial log if the type of the initial log is the screening type, obtaining a log to be uploaded, and uploading the log to be uploaded to the cloud server, wherein the terminal device and the edge server are in the same communication cluster.
11. The system of claim 10, wherein the edge server further comprises a storage management module;
the uploading module is further configured to send the received initial log sent by the terminal device to a storage management module of the edge server;
the storage management module is used for storing the initial log sent by the terminal equipment into a message queue through a production thread of the message queue;
the uploading module is further configured to obtain the initial log from the message queue through a consumption thread of the message queue;
the edge server also comprises a log query module and an index management module; the type of the initial log comprises a level of the initial log;
the uploading module is specifically configured to, if the initial log includes a log of which the level is error, obtain, by the log query module, an upper log of the log of which the level is error, and use the log of which the level is error and the upper log thereof as the log to be uploaded;
the log query module is configured to acquire information of the terminal device associated with the log of the rank error, and acquire the log above the log of the rank error according to the information of the terminal device and a correspondence between the terminal device and a log file acquired by the index management module, where the log of the rank error is generated by the terminal device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010617963.1A CN111835562B (en) | 2020-07-01 | 2020-07-01 | Log uploading method, log inquiring method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010617963.1A CN111835562B (en) | 2020-07-01 | 2020-07-01 | Log uploading method, log inquiring method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111835562A true CN111835562A (en) | 2020-10-27 |
| CN111835562B CN111835562B (en) | 2023-10-17 |
Family
ID=72899924
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010617963.1A Active CN111835562B (en) | 2020-07-01 | 2020-07-01 | Log uploading method, log inquiring method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111835562B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113360468A (en) * | 2021-05-12 | 2021-09-07 | 武汉联影医疗科技有限公司 | Log data uploading method and device, computer equipment and readable storage medium |
| CN113742031A (en) * | 2021-08-27 | 2021-12-03 | 北京百度网讯科技有限公司 | Node state information acquisition method and device, electronic equipment and readable storage medium |
| CN114745291A (en) * | 2020-12-23 | 2022-07-12 | 北京国双科技有限公司 | Abnormal data cloud method and device, electronic equipment and storage medium |
| CN114826774A (en) * | 2022-06-02 | 2022-07-29 | 天翼数字生活科技有限公司 | Terminal device log request control method, server and system |
| CN115378804A (en) * | 2022-08-19 | 2022-11-22 | 深圳前海环融联易信息科技服务有限公司 | Log acquisition method, device, equipment and medium based on cloud edge cooperation |
| CN116401275A (en) * | 2023-03-14 | 2023-07-07 | 宁波浙鼎教育科技有限公司 | Cloud data processing method, cloud data processing system, storage medium and intelligent terminal |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7155514B1 (en) * | 2002-09-12 | 2006-12-26 | Dorian Software Creations, Inc. | Apparatus for event log management |
| CN101969386A (en) * | 2010-11-09 | 2011-02-09 | 道有道(北京)科技有限公司 | Log acquisition device and log acquisition method |
| US20110040811A1 (en) * | 2009-08-17 | 2011-02-17 | International Business Machines Corporation | Distributed file system logging |
| CN105119752A (en) * | 2015-09-08 | 2015-12-02 | 北京京东尚科信息技术有限公司 | Distributed log acquisition method, device and system |
| CN105389314A (en) * | 2014-09-04 | 2016-03-09 | 中芯国际集成电路制造(上海)有限公司 | Log file query system and query method |
| CN105703932A (en) * | 2014-11-28 | 2016-06-22 | 广州华多网络科技有限公司 | Log recording and uploading method, log recording and reception method, terminal, and server |
| JP2016177594A (en) * | 2015-03-20 | 2016-10-06 | 株式会社リコー | Communication terminal device, communication system, communication method and program |
| CN106033322A (en) * | 2015-03-17 | 2016-10-19 | 北京元心科技有限公司 | Method and device for data storage |
| WO2016169145A1 (en) * | 2015-04-21 | 2016-10-27 | 中兴通讯股份有限公司 | Cloud terminal log processing method, device and system |
| CN107809337A (en) * | 2017-11-17 | 2018-03-16 | 深圳泉眼体育运营管理有限公司 | A kind of daily record method for uploading and device |
| CN109617726A (en) * | 2018-12-14 | 2019-04-12 | 深圳壹账通智能科技有限公司 | Error log acquisition method and server |
| CN110034610A (en) * | 2019-05-10 | 2019-07-19 | 国家电网有限公司 | A kind of transformer station measurement and control device redundancy configuration method and system |
| CN110198275A (en) * | 2018-03-28 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of flow control methods, system, server and storage medium |
| CN110995611A (en) * | 2019-12-20 | 2020-04-10 | 创盛视联数码科技（北京）有限公司 | Distributed current limiting method for high concurrency request |
| CN111190871A (en) * | 2019-12-18 | 2020-05-22 | 中移（杭州）信息技术有限公司 | Log generation method and device, computer equipment and storage medium |
| CN111352902A (en) * | 2020-02-28 | 2020-06-30 | 中国平安人寿保险股份有限公司 | Log processing method and device, terminal equipment and storage medium |
-
2020
- 2020-07-01 CN CN202010617963.1A patent/CN111835562B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7155514B1 (en) * | 2002-09-12 | 2006-12-26 | Dorian Software Creations, Inc. | Apparatus for event log management |
| US20110040811A1 (en) * | 2009-08-17 | 2011-02-17 | International Business Machines Corporation | Distributed file system logging |
| CN101969386A (en) * | 2010-11-09 | 2011-02-09 | 道有道(北京)科技有限公司 | Log acquisition device and log acquisition method |
| CN105389314A (en) * | 2014-09-04 | 2016-03-09 | 中芯国际集成电路制造(上海)有限公司 | Log file query system and query method |
| CN105703932A (en) * | 2014-11-28 | 2016-06-22 | 广州华多网络科技有限公司 | Log recording and uploading method, log recording and reception method, terminal, and server |
| CN106033322A (en) * | 2015-03-17 | 2016-10-19 | 北京元心科技有限公司 | Method and device for data storage |
| JP2016177594A (en) * | 2015-03-20 | 2016-10-06 | 株式会社リコー | Communication terminal device, communication system, communication method and program |
| WO2016169145A1 (en) * | 2015-04-21 | 2016-10-27 | 中兴通讯股份有限公司 | Cloud terminal log processing method, device and system |
| CN105119752A (en) * | 2015-09-08 | 2015-12-02 | 北京京东尚科信息技术有限公司 | Distributed log acquisition method, device and system |
| CN107809337A (en) * | 2017-11-17 | 2018-03-16 | 深圳泉眼体育运营管理有限公司 | A kind of daily record method for uploading and device |
| CN110198275A (en) * | 2018-03-28 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of flow control methods, system, server and storage medium |
| CN109617726A (en) * | 2018-12-14 | 2019-04-12 | 深圳壹账通智能科技有限公司 | Error log acquisition method and server |
| CN110034610A (en) * | 2019-05-10 | 2019-07-19 | 国家电网有限公司 | A kind of transformer station measurement and control device redundancy configuration method and system |
| CN111190871A (en) * | 2019-12-18 | 2020-05-22 | 中移（杭州）信息技术有限公司 | Log generation method and device, computer equipment and storage medium |
| CN110995611A (en) * | 2019-12-20 | 2020-04-10 | 创盛视联数码科技（北京）有限公司 | Distributed current limiting method for high concurrency request |
| CN111352902A (en) * | 2020-02-28 | 2020-06-30 | 中国平安人寿保险股份有限公司 | Log processing method and device, terminal equipment and storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114745291A (en) * | 2020-12-23 | 2022-07-12 | 北京国双科技有限公司 | Abnormal data cloud method and device, electronic equipment and storage medium |
| CN113360468A (en) * | 2021-05-12 | 2021-09-07 | 武汉联影医疗科技有限公司 | Log data uploading method and device, computer equipment and readable storage medium |
| CN113742031A (en) * | 2021-08-27 | 2021-12-03 | 北京百度网讯科技有限公司 | Node state information acquisition method and device, electronic equipment and readable storage medium |
| CN114826774A (en) * | 2022-06-02 | 2022-07-29 | 天翼数字生活科技有限公司 | Terminal device log request control method, server and system |
| CN114826774B (en) * | 2022-06-02 | 2024-01-30 | 天翼数字生活科技有限公司 | Log request control method, server side and system for terminal equipment |
| CN115378804A (en) * | 2022-08-19 | 2022-11-22 | 深圳前海环融联易信息科技服务有限公司 | Log acquisition method, device, equipment and medium based on cloud edge cooperation |
| CN116401275A (en) * | 2023-03-14 | 2023-07-07 | 宁波浙鼎教育科技有限公司 | Cloud data processing method, cloud data processing system, storage medium and intelligent terminal |
| CN116401275B (en) * | 2023-03-14 | 2023-11-14 | 宁波浙鼎教育科技有限公司 | Cloud data processing method, cloud data processing system, storage medium and intelligent terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111835562B (en) | 2023-10-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111835562B (en) | Log uploading method, log inquiring method and system | |
| CN111131379B (en) | Distributed flow acquisition system and edge calculation method | |
| US11573725B2 (en) | Object migration method, device, and system | |
| CN111522636B (en) | Application container adjusting method, application container adjusting system, computer readable medium and terminal device | |
| CN108287905B (en) | Method for extracting and storing network flow characteristics | |
| CN109460343A (en) | System exception monitoring method, device, equipment and storage medium based on log | |
| CN106027595A (en) | Access log processing method and system for CDN node | |
| CN102984244A (en) | Uploading system and uploading method of bill data | |
| CN111562889B (en) | Data processing method, device, system and storage medium | |
| CN106713378B (en) | Method and system for providing service by multiple application servers | |
| CN112689007B (en) | Resource allocation method, device, computer equipment and storage medium | |
| CN111522786A (en) | Log processing system and method | |
| CN112737817A (en) | Network slice resource dynamic partitioning method and device based on multi-parameter determination | |
| CN111064591B (en) | Data aggregation method, device, equipment, storage medium and system | |
| CN104869155A (en) | Data auditing method and device | |
| CN112698952A (en) | Unified management method and device for computing resources, computer equipment and storage medium | |
| CN116360954B (en) | Industrial Internet of things management and control method and system based on cloud edge cooperative technology | |
| CN116225609A (en) | Pod scheduling method, device and equipment of Kubernetes cluster and computer readable medium | |
| CN109783073A (en) | Bullet contracting method and device, the micro services, storage medium of application container | |
| CN112737980B (en) | Time-based network slice resource dynamic partitioning method and device | |
| CN112019604B (en) | Edge data transmission method and system | |
| CN113794282A (en) | Data processing method and device based on intelligent gateway and computer equipment | |
| CN113364651A (en) | Novel distributed network flow acquisition method | |
| CN117240734A (en) | Cloud edge cooperation method, cloud edge cooperation system, computer equipment and storage medium | |
| CN115499514A (en) | Data storage service access method, computing device and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |