CN110519077A - Log acquisition system, method, device, server and storage medium - Google Patents
Log acquisition system, method, device, server and storage medium Download PDFInfo
- Publication number
- CN110519077A CN110519077A CN201910684064.0A CN201910684064A CN110519077A CN 110519077 A CN110519077 A CN 110519077A CN 201910684064 A CN201910684064 A CN 201910684064A CN 110519077 A CN110519077 A CN 110519077A
- Authority
- CN
- China
- Prior art keywords
- log
- node
- information
- client
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000012545 processing Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 238000005303 weighing Methods 0.000 claims 1
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 11
- 238000006243 chemical reaction Methods 0.000 description 6
- 238000001914 filtration Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 208000033748 Device issues Diseases 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/0246—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application discloses a log collection system, which comprises a log server and a first preset number of node devices, wherein each node device is provided with a log client; the log client is used for collecting logs of corresponding node equipment according to a log collection instruction issued by the log server, obtaining node logs, converting the node logs into log information in a preset format and uploading the log information to the log server; the log server is used for issuing the log collection instruction to each log client and receiving the log information uploaded by each log client; the log collection system can collect logs of different devices, platforms, applications and systems, unification of log collection is achieved, and manageability of log collection is improved. The application also discloses a log collection method, a log collection device, a server and a computer readable storage medium, which have the beneficial effects.
Description
Technical field
This application involves data center management field, in particular to a kind of Log Collect System further relates to a kind of log and adopts
Set method, device, server and computer readable storage medium.
Background technique
With the fast development of internet, the technologies iteratively faster such as mobile Internet, cloud computing, big data, Internet of Things produces
A large amount of data and log have been given birth to, meanwhile, on above-mentioned technology platform, run such as search engine, magnanimity cloud service, electricity
The IT platforms such as sub- commercial affairs, video website, social networks, shared bicycle, can equally generate massive logs daily；Moreover, for big
For medium-sized enterprise, data center includes all kinds of different hardware devices, such as server, storage, enterprise router and server
The types of applications of upper operation can also generate a large amount of logs, therefore, reasonably acquired to log and processing seems more heavy
It wants.
However, log collection method in the prior art is both for particular device or particular platform, such as VMware
Log collection process can only be acquired the log in specified platform, the log of other platforms can only be by respectively right by therefrom
The log collection program answered carries out log collection, that is to say, that is not provided with a kind of unified log collection in the prior art
Method, can not from different physical equipments, virtualized environment, log collection is realized in, lead to subsequent daily record data as a result,
The operating process such as summarize, analyze, showing all there is higher complexity.
Therefore, a kind of unified log collection method how is provided, to realize distinct device, platform, application, system simultaneously
Log collection, the manageability for improving log collection is those skilled in the art's urgent problem to be solved.
Summary of the invention
The purpose of the application is to provide a kind of Log Collect System, which can be to different equipment, flat
Platform, application, system carry out log collection, realize the unitized of log collection, improve the manageability of log collection；This Shen
Another object please is to provide a kind of log collection method, device, server and computer readable storage medium, it may have on
State beneficial effect.
In order to solve the above technical problems, this application provides a kind of Log Collect System, the Log Collect System includes
Log server and the first preset quantity node device, wherein be provided with log client on each node device；
The log client, the log collection instruction for being issued according to the log server set corresponding node
It is standby to carry out log collection, node log is obtained, the node log is converted to the log information of preset format, and by the day
Will information is uploaded to the log server；
The log server is instructed for issuing the log collection to each log client, and is received each
The log information that the log client uploads.
Preferably, the Log Collect System further include:
Second preset quantity log converter, for obtaining the day of specified Log Types from each log client
Will information, and the log information of the specified Log Types is uploaded to the log server.
Preferably, the log server is provided with third preset quantity calculate node, wherein each calculating section
Point is for carrying out log processing to the log information.
Preferably, the log server is provided with internal load balanced device, for counting working as each calculate node
Preceding operation load, and load balancing is carried out to each calculate node according to each current operating load.
In order to solve the above technical problems, this application provides a kind of log collection method, the log collection method includes:
Receive log collection instruction；
Log collection instruction is sent to the corresponding log client of each node device, so that each log
Client carries out log collection to corresponding node device, obtains node log, the node log is converted to preset format
Log information；
Receive the log information that each log client uploads.
Preferably, the log information that the node log is converted to preset format, comprising:
Determine the attribute information of the node log；
Corresponding log parsing strategy is transferred according to the attribute information；
Parsing is carried out to the node log by log parsing strategy and format is converted, obtains the preset format
Log information.
Preferably, the log information for receiving each log client and uploading, comprising:
The log information that each log client uploads is received using syslog agreement and/or standard api interface.
In order to solve the above technical problems, present invention also provides a kind of log acquisition apparatus, the log acquisition apparatus packet
It includes:
Command reception module, for receiving log collection instruction；
Instruction issues module, for log collection instruction to be sent to the corresponding log client of each node device
End obtains node log, by the node day so that each log client carries out log collection to corresponding node device
Will is converted to the log information of preset format；
Log acquisition module, the log information uploaded for receiving each log client.
In order to solve the above technical problems, present invention also provides a kind of server, the server includes:
Memory, for storing computer program；
Processor, the step of any one of the above log collection method is realized when for executing the computer program.
In order to solve the above technical problems, the computer can present invention also provides a kind of computer readable storage medium
It reads to be stored with computer program on storage medium, the computer program realizes any one of the above log when being executed by processor
The step of acquisition method.
A kind of Log Collect System provided herein, including log server and the first preset quantity node are set
It is standby, wherein to be provided with log client on each node device；The log client, for being taken according to the log
The log collection instruction that business device issues carries out log collection to corresponding node device, node log is obtained, by the node day
Will is converted to the log information of preset format, and the log information is uploaded to the log server；The log services
Device is instructed for issuing the log collection to each log client, and receives what each log client uploaded
The log information.
As it can be seen that Log Collect System provided herein, by adding corresponding log visitor on all kinds of node devices
Family end for acquiring generated log information in itself task processes, and is uniformly uploaded to log server, and then i.e.
Can be realized by log server all kinds of node devices unified log collection and subsequent log processing, the Log Collect System
It is not limited by node device type, different physical equipments, different platform, different application, the not logs such as homologous ray may be implemented
The acquisition of information, and then cross-platform log collection is realized, the unitized of log collection is realized, log collection is effectively increased
Manageability.
A kind of log collection method, device, server and computer readable storage medium provided herein, has
There is above-mentioned beneficial effect, details are not described herein.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of structural schematic diagram of Log Collect System provided herein；
Fig. 2 is a kind of flow diagram of log collection method provided herein；
Fig. 3 is a kind of flow diagram of log conversion method provided herein；
Fig. 4 is the structural schematic diagram of another Log Collect System provided herein；
Fig. 5 is a kind of structural schematic diagram of log acquisition apparatus provided herein；
Fig. 6 is a kind of structural schematic diagram of server provided herein.
Specific embodiment
The core of the application is to provide a kind of Log Collect System, which can be to different equipment, flat
Platform, application, system carry out log collection, realize the unitized of log collection, improve the manageability of log collection；This Shen
Another core please is to provide a kind of log collection method, device, server and computer readable storage medium, it may have on
State beneficial effect.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is
Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art
All other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Referring to FIG. 1, Fig. 1 is a kind of structural schematic diagram of Log Collect System provided herein, the log collection
System may include log server 1 and the first preset quantity node device 2, wherein be provided with day on each node device 2
Will client 3；
Log client 3, for being issued according to log server 1 log collection instruction to corresponding node device 2 into
Row log collection obtains node log, node log is converted to the log information of preset format, and log information is uploaded to
Log server 1；
Log server 1, for issuing log collection instruction, and each log client 3 of reception to each log client 3
The log information of upload.
Log Collect System provided herein includes log server 1 and the first preset quantity node device 2, often
Log client 3 is provided on a node device 2, wherein log server 1 is mainly used for realizing under log collection instruction
The acquisition of the log information of hair and each node device 2, log client 3 are mainly used for carrying out its corresponding node device 2
Log collection and journal format conversion.
Specifically, the log collection can be instructed and be distributed to when log server 1 gets log collection instruction
The corresponding log client 3 of each node device 2, log client 3 can be instructed according to log collection and be corresponded to itself as a result,
Node device 2 carry out log collection, obtain corresponding node log, due to be finally realized by log server 1 it is a variety of
Log collection, storage and the analysis processing of the node device 2 of type, therefore, collect node log in log client 3
Afterwards, it can be converted into the log information of preset format, to guarantee the system of log information type that log server 1 finally obtains
One property.
It is understood that since this system can realize cross-platform log collection, above-mentioned node device 2 it is specific
Type is not unique, can be any kind of physical equipment, virtual unit, application platform, application and system, specifically such as
VSphere host, Linux host, Windows host, cloud computing platform, big data platform, distributed memory system, video network
It stands.Certainly, the particular content of above-mentioned log information is not also unique, can be node device 2 in any task processes
Generated all kinds of log informations, such as temporal information, store path, storage information, fault message.In addition, above-mentioned first is pre-
If the specific value of quantity does not influence the implementation of the technical program equally, the application is not limited this.
As a kind of preferred embodiment, which can also include the second preset quantity log converter,
For obtaining the log information of specified Log Types from each log client 3, and will be on the log information of specified Log Types
Reach log server 1.
Log Collect System provided by the embodiment of the present application, is additionally arranged log converter, for realizing specified log class
The acquisition and upload of the log information of type.Firstly, in changing Log Collect System, the quantity of log converter, i.e., above-mentioned second
The specific value of preset quantity is not unique, can according to actual needs, as log information specific classification carry out quantity setting, one
Kind Log Types correspond to a log converter.Secondly, each log converter can be with one or more log clients 3
It is connected, to realize the acquisition of the log information of specified Log Types, further, the day for the specified Log Types that will acquire
Will information is uploaded to log server 1.The log information of specified Log Types is realized by setting log converter as a result,
Acquisition, be equivalent to during log collection just complete log classification, be effectively reduced the burden of log server 1, just
The operating processes such as the storage of the log in subsequent process, analysis and display are absorbed in log server 1.
Preferably, above-mentioned log server 1 is provided with third preset quantity calculate node, wherein each calculate node
For carrying out log processing to log information.
Log server 1 provided by the embodiment of the present application, after multiple calculate nodes realization log informations can be set
Continuous processing carries out parallel processing to log information using multiple calculate nodes, effectively increases the treatment effeciency of log information,
It is of course also possible to realize the processing of the log information of different Log Types using different calculate nodes, not uniquely.Equally
, for the particular number of above-mentioned calculate node, i.e., the specific value of above-mentioned third preset quantity, by technical staff according to reality
Demand is set, and the application does not limit this.
Preferably, above-mentioned log server 1 is provided with internal load balanced device, for counting the current fortune of each calculate node
Row load, and load balancing is carried out to each calculate node according to each current operating load.
Log Collect System provided by the embodiment of the present application, log server 1 are provided with internal load balanced device, main
To guarantee the stable operation of log server 1 for realizing the load balancing of each calculate node.Specifically, negative using inside
It carries balanced device and real-time statistics is carried out to the current operating load of each calculate node, when the current operating load for finding certain calculate node
When beyond level threshold value, the fractional load equilibrium of its current operating load can be loaded in lesser calculate nodes to other, it is real
The load balancing of existing calculate node.Wherein, the specific value of above-mentioned standard threshold value is set according to the specific performance of calculate node
It is fixed.
Log Collect System provided herein, by adding corresponding log client on all kinds of node devices,
For acquiring generated log information in itself task processes, and it is uniformly uploaded to log server, and then can led to
Cross log server realize all kinds of node devices unified log collection and subsequent log processing, the Log Collect System not by
Different physical equipments, different platform, different application, the not log informations such as homologous ray may be implemented in the limitation of node device type
Acquisition, and then realize cross-platform log collection, realize the unitized of log collection, effectively increase log collection can
It is managerial.
Based on above-mentioned Log Collect System, referring to FIG. 2, Fig. 2 is a kind of log collection method provided herein
Flow diagram, the log collection method can include:
S101: log collection instruction is received；
S102: log collection instruction is sent to the corresponding log client of each node device, so that each log client
Log collection is carried out to corresponding node device, node log is obtained, node log is converted to the log information of preset format；
It is real that log collection method provided by the embodiment of the present application is based on Log Collect System provided by a upper embodiment
Existing, Log Collect System includes log server and the first preset quantity node device, is provided with day on each node device
Will client.
Firstly, log server receives log collection instruction, log collection instruction can be based on front end by technical staff
Input can also be based on preset trigger condition, such as timer command automated tos respond to, it is seen then that the acquisition of above-mentioned log collection instruction
Mode is not unique, and the application does not limit this.Further, log collection instruction is issued to each node by log server
The corresponding log client of equipment is instructed according to log collection by log client and carries out log to itself corresponding node device
Acquisition, obtains corresponding node log, and then be converted into the log information of preset format, is finally uploaded to log information
Log server, so that the uniformity of the type for the log information that log server finally obtains be effectively ensured.
In addition, log filtering policy can also be set when carrying out node log acquisition, to utilize log filtering policy pair
Collected all node logs are filtered, and are the behaviour of subsequent log analysis to realize the selectivity acquisition of log information
Bigger convenience is provided as process.
For the format conversion process of log, the embodiment of the present application provides a kind of more concrete implementation method, please join
Examine Fig. 3, such as 3 be journal format conversion method provided herein flow diagram, it is above-mentioned node log is converted to it is pre-
If the log information of format, may include:
S201: the attribute information of node log is determined；
S202: corresponding log is transferred according to attribute information and parses strategy；
S203: parsing is carried out to node log by log parsing strategy and format is converted, obtains the log of preset format
Information.
Specifically, since node log derives from different node devices, and the type of node log, attribute are not only
One, therefore, for the node log of different attribute, different types of log parsing strategy can be preset and realize journal format conversion.
As a result, log client can determine its attribute information after the node log for collecting corresponding node equipment, and according to the category
Property information transfer corresponding log parsing strategy, finally parse that strategy carries out parsing to node log and format turns using the log
It changes, to obtain the log information of above-mentioned preset format.Wherein, the particular content of above-mentioned node log attribute information is not only
One, it is specifically as follows Log Types information, acquisition source-information, acquisition time information etc..
S103: the log information that each log client uploads is received.
In this step, the log information that log server can receive that each log client uploads so far completes
The log collection of different node devices.Further, log server can execute corresponding storage, analysis to these log informations
The operating processes such as processing, display.
Preferably, the log information that each log client of above-mentioned reception uploads, may include: using syslog agreement and/
Or standard api interface receives the log information that each log client uploads.
For the method for uploading of log information, this application provides more concrete implementation methods, i.e., are assisted based on syslog
View and/or standard api interface are realized.Wherein, syslog agreement is a kind of standard for transmitting log that Linux system is supported, is
A kind of agreement of industrial standard, can be used to the log of recording equipment；Standard api interface (Application Program
Interface, application programming interfaces) be according to other servers, system, application definition a kind of coffret, specifically stay
To a calling interface of application program.In Log Collect System provided herein, both the above implementation is deposited simultaneously
Convenient for selecting the upload mode of different log informations according to the actual situation.
In addition, the acquisition mode of the log information for different node devices, can be realized by fixed cycle operator, specifically may be used
To be to issue log collection instruction by log server timing, it is also possible to that timing script is arranged in log client, to node
Log is acquired in real time, and uploads to log server according to prefixed time interval.
Log collection method provided by the embodiment of the present application, by adding corresponding log visitor on all kinds of node devices
Family end for acquiring generated log information in itself task processes, and is uniformly uploaded to log server, and then i.e.
Can be realized by log server all kinds of node devices unified log collection and subsequent log processing, the Log Collect System
It is not limited by node device type, different physical equipments, different platform, different application, the not logs such as homologous ray may be implemented
The acquisition of information, and then cross-platform log collection is realized, the unitized of log collection is realized, log collection is effectively increased
Manageability.
On the basis of the various embodiments described above, the application embodiment provides a kind of more specifically log collection system
System, referring to FIG. 4, Fig. 4 is the structural schematic diagram of another Log Collect System provided herein.
Firstly, carrying out the deployment of log client.Specifically, data center collector journal in need hardware
Or in system, including vSphere host, storage, network and application component, equal deployment configuration log client, such as Fig. 4 institute
Show Linux host deployments client, Windows host deployments client, deploying client of other equipment etc.；Meanwhile it disposing
Log Router collects same type of log information for unified, and is uploaded to log server.In each log visitor
On the end of family, statistics needs the log information attribute collected, such as Log Types, log path suffix, collection rule, alarm level mistake
Filter, comprising the corresponding log filtering policies of information settings such as/filtered fields and log parsing strategy, to realize the selection of log
Property acquisition and format conversion.In addition, in the Log Collect System provided by the embodiment of the present application, for collected node day
Will can uniformly be converted to JSON format string.
Further, the upload mode of log information is set.Specifically, the Log Collect System is supported active push, is adjusted
It is uploaded with the timing that server end standard interface two ways carries out log information, log client and log converter can be with
Log information is uploaded to by log server by syslog agreement or standard api interface according to prefixed time interval.
Finally, carrying out the setting of log server.Specifically, multiple calculating sections can be arranged inside log server
Point, node 1 as shown in Figure 4, node 2 ... node x, for collecting multiple log clients parallel, Log Router reports
Log information；Internal load balancing device can also be set, for realizing the load balancing of each calculate node, guarantee log collection
The stable operation of efficiency and log server.In addition, log server supports the filtering function and multiple types of log information
Display function, for example, can according to log source, log field, time etc. to log information carry out screening and filtering, Ke Yitong
It crosses the forms such as list or line chart to be shown log information, i.e., the log server can be according to actual needs to all kinds of days
Will information carries out selective displaying, and more convenient technical staff carries out log access and fault location.
It can be seen that Log Collect System provided by the embodiment of the present application, in towards medium-and-large-sized enterprise data center,
The information such as different hardware, system, the running log of application program, alarm can be carried out unification by the log collection mode of offer
It collects and shows, and provide detailed visualization interface, to check that influence host, storage assembly or other data centers are hard
Event, warning information of part etc..In addition, the components such as log client, Log Router can also be extending transversely, effectively to meet number
According to the portfolio and data flow of center rapid growth, it is greatly improved the reliability of log collection, scalability and can manages
Rationality.
Log collection method provided by the embodiment of the present application, by adding corresponding log visitor on all kinds of node devices
Family end for acquiring generated log information in itself task processes, and is uniformly uploaded to log server, and then i.e.
Can be realized by log server all kinds of node devices unified log collection and subsequent log processing, the Log Collect System
It is not limited by node device type, different physical equipments, different platform, different application, the not logs such as homologous ray may be implemented
The acquisition of information, and then cross-platform log collection is realized, the unitized of log collection is realized, log collection is effectively increased
Manageability.
To solve the above problems, referring to FIG. 5, the structure that Fig. 5 is a kind of log acquisition apparatus provided herein is shown
It is intended to, the log acquisition apparatus can include:
Command reception module 100, for receiving log collection instruction；
Instruction issues module 200, for log collection instruction to be sent to the corresponding log client of each node device, with
So that each log client is carried out log collection to corresponding node device, obtains node log, node log is converted to default
The log information of format；
Log acquisition module 300, the log information uploaded for receiving each log client.
As it can be seen that log acquisition apparatus provided by the embodiment of the present application, corresponding by adding on all kinds of node devices
Log client for acquiring generated log information in itself task processes, and is uniformly uploaded to log server,
And then it can realize that the unified log collection of all kinds of node devices and subsequent log processing, the log are adopted by log server
Collecting system is not limited by node device type, and different physical equipments, different platform, different application, not homologous ray may be implemented
The acquisition of equal log informations, and then cross-platform log collection is realized, the unitized of log collection is realized, day is effectively increased
The manageability of will acquisition.
As a kind of preferred embodiment, above-mentioned log acquisition module 300 can be specifically used for utilizing syslog agreement and/or mark
Quasi- api interface receives the log information that each log client uploads.
Above method embodiment is please referred to for the introduction of device provided by the present application, this will not be repeated here by the application.
To solve the above problems, referring to FIG. 6, Fig. 6 be a kind of server provided herein structural schematic diagram, should
Server can include:
Memory 10, for storing computer program；
Processor 20, realizes following steps when for executing computer program:
Receive log collection instruction；Log collection instruction is sent to the corresponding log client of each node device, so that
Each log client carries out log collection to corresponding node device, obtains node log, node log is converted to default lattice
The log information of formula；Receive the log information that each log client uploads.
Above method embodiment is please referred to for the introduction of server provided by the present application, this will not be repeated here by the application.
To solve the above problems, present invention also provides a kind of computer readable storage medium, the computer-readable storage
It is stored with computer program on medium, can be realized when computer program is executed by processor following rapid:
Receive log collection instruction；Log collection instruction is sent to the corresponding log client of each node device, so that
Each log client carries out log collection to corresponding node device, obtains node log, node log is converted to default lattice
The log information of formula；Receive the log information that each log client uploads.
The computer readable storage medium may include: USB flash disk, mobile hard disk, read-only memory (Read-Only
Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. is various to deposit
Store up the medium of program code.
Above method embodiment is please referred to for the introduction of computer readable storage medium provided by the present application, the application exists
This is not repeated them here.
Each embodiment is described in a progressive manner in specification, the highlights of each of the examples are with other realities
The difference of example is applied, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
Speech, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part illustration
.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Above to Log Collect System provided herein, method, apparatus, server and computer-readable storage medium
Matter is described in detail.Specific examples are used herein to illustrate the principle and implementation manner of the present application, above
The explanation of embodiment is merely used to help understand the present processes and its core concept.It should be pointed out that for the art
Those of ordinary skill for, under the premise of not departing from the application principle, can also to the application carry out it is several improvement and repair
Decorations, these improvement and modification also fall into the protection scope element of the claim of this application.
Claims (10)
1. a kind of Log Collect System, which is characterized in that including log server and the first preset quantity node device,
In, log client is provided on each node device；
The log client, for being issued according to the log server log collection instruction to corresponding node device into
Row log collection obtains node log, the node log is converted to the log information of preset format, and the log is believed
Breath is uploaded to the log server；
The log server is instructed for issuing the log collection to each log client, and is received each described
The log information that log client uploads.
2. Log Collect System as described in claim 1, which is characterized in that further include:
Second preset quantity log converter, for obtaining the log letter of specified Log Types from each log client
Breath, and the log information of the specified Log Types is uploaded to the log server.
3. Log Collect System as claimed in claim 1 or 2, which is characterized in that it is pre- that the log server is provided with third
If quantity calculate node, wherein each calculate node is used to carry out log processing to the log information.
4. Log Collect System as claimed in claim 3, which is characterized in that it is equal that the log server is provided with internal load
Weighing apparatus, for counting the current operating load of each calculate node, and according to each current operating load to each meter
Operator node carries out load balancing.
5. a kind of log collection method characterized by comprising
Receive log collection instruction；
Log collection instruction is sent to the corresponding log client of each node device, so that each log client
End carries out log collection to corresponding node device, obtains node log, the node log is converted to the day of preset format
Will information；
Receive the log information that each log client uploads.
6. log collection method as claimed in claim 5, which is characterized in that described that the node log is converted to default lattice
The log information of formula, comprising:
Determine the attribute information of the node log；
Corresponding log parsing strategy is transferred according to the attribute information；
Parsing is carried out to the node log by log parsing strategy and format is converted, obtains the day of the preset format
Will information.
7. such as log collection method described in claim 5 or 6, which is characterized in that described to receive in each log client
The log information passed, comprising:
The log information that each log client uploads is received using syslog agreement and/or standard api interface.
8. a kind of log acquisition apparatus characterized by comprising
Command reception module, for receiving log collection instruction；
Instruction issues module, for log collection instruction to be sent to the corresponding log client of each node device,
So that each log client carries out log collection to corresponding node device, node log is obtained, by the node log
Be converted to the log information of preset format；
Log acquisition module, the log information uploaded for receiving each log client.
9. a kind of server, which is characterized in that further include:
Memory, for storing computer program；
Processor realizes the log collection side as described in claim 5 to 7 any one when for executing the computer program
The step of method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the log collection method as described in claim 5 to 7 any one when the computer program is executed by processor
The step of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910684064.0A CN110519077A (en) | 2019-07-26 | 2019-07-26 | Log acquisition system, method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910684064.0A CN110519077A (en) | 2019-07-26 | 2019-07-26 | Log acquisition system, method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110519077A true CN110519077A (en) | 2019-11-29 |
Family
ID=68624193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910684064.0A Pending CN110519077A (en) | 2019-07-26 | 2019-07-26 | Log acquisition system, method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110519077A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110990335A (en) * | 2019-12-06 | 2020-04-10 | 深圳前海微众银行股份有限公司 | Log archiving method, device, equipment and computer readable storage medium |
| CN110995500A (en) * | 2019-12-16 | 2020-04-10 | 深圳市网心科技有限公司 | Node log management and control method, system and related components |
| CN111212115A (en) * | 2019-12-23 | 2020-05-29 | 恩亿科(北京)数据科技有限公司 | Method and device for realizing hybrid cloud management, computer storage medium and terminal |
| CN111625382A (en) * | 2020-05-21 | 2020-09-04 | 浪潮电子信息产业股份有限公司 | Server fault diagnosis method, device, equipment and medium |
| CN112328463A (en) * | 2020-11-27 | 2021-02-05 | 中国农业银行股份有限公司 | Log monitoring method and device |
| CN112350861A (en) * | 2020-10-29 | 2021-02-09 | 深圳市广和通无线股份有限公司 | Log obtaining method and device, computer equipment and storage medium |
| CN113297148A (en) * | 2021-06-18 | 2021-08-24 | 杭州安恒信息安全技术有限公司 | Method, device and equipment for acquiring service log data and readable storage medium |
| CN113342619A (en) * | 2021-07-06 | 2021-09-03 | 大象慧云信息技术有限公司 | Log monitoring method and system, electronic device and readable medium |
| CN113381907A (en) * | 2021-06-04 | 2021-09-10 | 挂号网（杭州）科技有限公司 | Log collection method and device, electronic equipment and storage medium |
| CN114490164A (en) * | 2022-04-02 | 2022-05-13 | 深圳市云视机器人有限公司 | Log collection method, system, device and computer storage medium |
| CN118277211A (en) * | 2024-06-03 | 2024-07-02 | 霖久智慧(广东)科技有限公司 | Log management method, system, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105589856A (en) * | 2014-10-21 | 2016-05-18 | 阿里巴巴集团控股有限公司 | Log data processing method and log data processing system |
| US20170177628A1 (en) * | 2015-12-17 | 2017-06-22 | Alibaba Group Holding Limited | Log fragmentation method and apparatus |
| CN109165193A (en) * | 2018-07-27 | 2019-01-08 | 阿里巴巴集团控股有限公司 | Storage method, device, client and the server of daily record data |
-
2019
- 2019-07-26 CN CN201910684064.0A patent/CN110519077A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105589856A (en) * | 2014-10-21 | 2016-05-18 | 阿里巴巴集团控股有限公司 | Log data processing method and log data processing system |
| US20170177628A1 (en) * | 2015-12-17 | 2017-06-22 | Alibaba Group Holding Limited | Log fragmentation method and apparatus |
| CN109165193A (en) * | 2018-07-27 | 2019-01-08 | 阿里巴巴集团控股有限公司 | Storage method, device, client and the server of daily record data |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110990335A (en) * | 2019-12-06 | 2020-04-10 | 深圳前海微众银行股份有限公司 | Log archiving method, device, equipment and computer readable storage medium |
| CN110995500A (en) * | 2019-12-16 | 2020-04-10 | 深圳市网心科技有限公司 | Node log management and control method, system and related components |
| CN111212115A (en) * | 2019-12-23 | 2020-05-29 | 恩亿科(北京)数据科技有限公司 | Method and device for realizing hybrid cloud management, computer storage medium and terminal |
| CN111625382B (en) * | 2020-05-21 | 2022-06-10 | 浪潮电子信息产业股份有限公司 | Server fault diagnosis method, device, equipment and medium |
| CN111625382A (en) * | 2020-05-21 | 2020-09-04 | 浪潮电子信息产业股份有限公司 | Server fault diagnosis method, device, equipment and medium |
| CN112350861A (en) * | 2020-10-29 | 2021-02-09 | 深圳市广和通无线股份有限公司 | Log obtaining method and device, computer equipment and storage medium |
| CN112350861B (en) * | 2020-10-29 | 2023-06-20 | 深圳市广和通无线股份有限公司 | Log acquisition method, device, computer equipment and storage medium |
| CN112328463A (en) * | 2020-11-27 | 2021-02-05 | 中国农业银行股份有限公司 | Log monitoring method and device |
| CN113381907A (en) * | 2021-06-04 | 2021-09-10 | 挂号网（杭州）科技有限公司 | Log collection method and device, electronic equipment and storage medium |
| CN113297148A (en) * | 2021-06-18 | 2021-08-24 | 杭州安恒信息安全技术有限公司 | Method, device and equipment for acquiring service log data and readable storage medium |
| CN113297148B (en) * | 2021-06-18 | 2023-06-06 | 杭州安恒信息安全技术有限公司 | Method, device and equipment for collecting service log data and readable storage medium |
| CN113342619A (en) * | 2021-07-06 | 2021-09-03 | 大象慧云信息技术有限公司 | Log monitoring method and system, electronic device and readable medium |
| CN114490164A (en) * | 2022-04-02 | 2022-05-13 | 深圳市云视机器人有限公司 | Log collection method, system, device and computer storage medium |
| CN118277211A (en) * | 2024-06-03 | 2024-07-02 | 霖久智慧(广东)科技有限公司 | Log management method, system, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110519077A (en) | Log acquisition system, method, device, server and storage medium | |
| US11924240B2 (en) | Mechanism for identifying differences between network snapshots | |
| US20160359701A1 (en) | Parallel coordinate charts for flow exploration | |
| US8478800B1 (en) | Log streaming facilities for computing applications | |
| CN105893205A (en) | Method and system for monitoring containers created based on docker | |
| US8265973B2 (en) | Analytic-based scaling of information technology resources | |
| CN107508722B (en) | Service monitoring method and device | |
| US20140297826A1 (en) | System and method for big data aggregation in sensor network | |
| US20200169470A1 (en) | Network migration assistant | |
| Trihinas et al. | Monitoring elastically adaptive multi-cloud services | |
| CN111459986B (en) | Data computing system and method | |
| CN110719194B (en) | Network data analysis method and device | |
| US10044820B2 (en) | Method and system for automated transaction analysis | |
| CN110661807A (en) | Automatic acquisition method and device for IPv6 address | |
| US20140250332A1 (en) | Log file reduction according to problem-space network topology | |
| US11716352B2 (en) | Application protectability schemes for enterprise applications | |
| CN112380091A (en) | Service operation condition monitoring method and device and related equipment | |
| CN107480189A (en) | A kind of various dimensions real-time analyzer and method | |
| WO2020211351A1 (en) | Method and device for obtaining external data by using crawler | |
| US9912545B2 (en) | High performance topology resolution for non-instrumented nodes | |
| US11895156B2 (en) | Securing network resources from known threats | |
| CN106302020B (en) | Network bandwidth statistical method and device | |
| US11463483B2 (en) | Systems and methods for determining effectiveness of network segmentation policies | |
| CN109101636A (en) | A kind of method, apparatus and system carrying out data acquisition in cloud by visual configuration | |
| CN107147542A (en) | A kind of information generating method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191129 |