CN108733546A - A kind of log collection method, device and equipment - Google Patents
A kind of log collection method, device and equipment Download PDFInfo
- Publication number
- CN108733546A CN108733546A CN201810285017.4A CN201810285017A CN108733546A CN 108733546 A CN108733546 A CN 108733546A CN 201810285017 A CN201810285017 A CN 201810285017A CN 108733546 A CN108733546 A CN 108733546A
- Authority
- CN
- China
- Prior art keywords
- file
- event
- journal file
- journal
- fileinfo
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This specification embodiment discloses log collection method, device and equipment, the method includes：The Log Directory for needing to monitor is preset, specified journal file and Log Directory are monitored；The corresponding fileinfo of class record of event is changed according to the file of the journal file received；Wherein, the type of file variation event includes：File removes event, file moves into event and file modification event, fileinfo include：Old file name, new filename, document misregistration amount and cookie；If above three event be based on the same former journal file in same catalogue (i.e., identical old file name) occur, then understand that the journal file has carried out daily record rotation, further, according to document misregistration amount, to acquiring daily record data in the corresponding former journal file of new filename.
Description
Technical field
This specification is related to a kind of field of computer technology more particularly to log collection method, device and equipment.
Background technology
In application system log management, individual log file is excessive in order to prevent, and application system can be according to certain rule
Round-robin processing then is carried out to the journal file of generation.For example, the database journal dal-digest.log of system, in rotation,
By former journal file renamed as dal-digest.log.1, an empty dal-digest.log is then generated for recording it
Daily record data afterwards.This daily record rotation mechanism does not have that fixed rotation is regular, for example, can according to the size of journal file into
Row rotation, can also be rotated according to turnaround time.In addition, the rule of the journal file renaming after rotation is also not fixed,
For example, renaming, the serial number renaming that can also be ranked up according to file size according to file can be carried out according to the time.
In the prior art, the mode that automatic regular polling is usually used carries out log collection, needs to pre-set to be collected
Journal file catalogue and file format.But since the rule of daily record rotation and the rule of journal file renaming be not solid
It is fixed, it needs that corresponding collection rule is respectively set for different rotation rule and Rename rule.If in a log collection
After, journal file originally has new daily record data write-in, daily record rotation then has occurred, then in timing wheel next time
When asking acquisition, the new journal file after log collection program will rotate daily record is acquired.
Based on the prior art, it is desirable to be able to efficiently, accurately carry out the scheme of log collection.
Invention content
This specification embodiment provides log collection method, device and equipment, for solving following technical problem：It needs
It can efficiently, accurately carry out the scheme of log collection.
In order to solve the above technical problems, what this specification embodiment was realized in：
A kind of log collection method that this specification embodiment provides, including：
The file for obtaining journal file changes event；
The type that event is changed according to the file, records the corresponding fileinfo of the journal file, wherein the text
Part information includes：The journal file changes front and back fileinfo and changes label information；
Based on the fileinfo, the data of the journal file are acquired.
A kind of log acquisition apparatus that this specification embodiment provides, including：
Acquisition module, the file for obtaining journal file change event；
Logging modle changes the type of event according to the file, records the fileinfo of the corresponding journal file,
Wherein, the fileinfo includes：The journal file changes front and back fileinfo and changes label information；
Acquisition module is based on the fileinfo, acquires the data of the journal file.
The a kind of electronic equipment that this specification embodiment provides, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by described at least one
A processor executes, so that at least one processor can：
The file for obtaining journal file changes event；
The type that event is changed according to the file, records the corresponding fileinfo of the journal file, wherein the text
Part information includes：The journal file changes front and back fileinfo and changes label information；
Based on the fileinfo, the data of the journal file are acquired.
Above-mentioned at least one technical solution that this specification embodiment uses can reach following advantageous effect：
By way of being monitored to specified Log Directory, if monitoring in same catalogue for same daily record text
Part has occurred file and removes event, file immigration event and file modification event, then it is believed that the journal file occurrence log wheel
Turn；Further, former journal file is determined according to new filename, daily record is carried out further according to the document misregistration amount in former journal file
The acquisition of data may be implemented to be acquired for the journal file rotated, repeated acquisition when avoiding poll acquisition
Problem can improve the efficiency of log collection；The acquisition of daily record data can also be carried out in time after occurrence log rotation, it can
To effectively prevent the loss of daily record data；By the way of being acquired based on event, without the concern for rotation rule and daily record
Rename rule can simplify the rule of log collection, reduce the difficulty of log collection.
Description of the drawings
In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
Some embodiments described in this specification, for those of ordinary skill in the art, in not making the creative labor property
Under the premise of, other drawings may also be obtained based on these drawings.
Fig. 1 is the schematic diagram for the Log Collect System that the scheme of this specification is related under practical application scene；
Fig. 2 is a kind of flow diagram for log collection method that this specification embodiment provides；
Fig. 3 is the method schematic diagram for the log collection based on document misregistration amount that this specification embodiment provides；
Fig. 4 is the flow diagram of a kind of daily record monitoring and gatherer process that this specification embodiment provides；
Fig. 5 is a kind of structural schematic diagram for log acquisition apparatus that this specification embodiment provides.
Specific implementation mode
In order to make those skilled in the art more fully understand the technical solution in this specification, below in conjunction with this explanation
Attached drawing in book embodiment is clearly and completely described the technical solution in this specification embodiment, it is clear that described
Embodiment be merely a part but not all of the embodiments of the present application.Based on this specification embodiment, this field
The every other embodiment that those of ordinary skill is obtained without creative efforts, should all belong to the application
The range of protection.
Fig. 1 is a kind of schematic diagram for the Log Collect System that the scheme of this specification is related under practical application scene.Root
Specified journal file and Log Directory are monitored according to preset Log Directory, if log acquisition server receives file
Removal event, then record cookie and old file name；Further, if receiving file moves into event, judge to move in file
Except whether the cookie moved into list of thing with file records identical the cookie in list of thing, if identical, record
Cookie, old file name and new filename；Finally, if receiving file modification event, old file name and document misregistration are recorded
Amount；If above three event is to be directed to same old file name, then it is assumed that the journal file occurrence log rotates；According to record
Document misregistration amount and daily record rotation after new filename, acquire daily record data from former journal file.Daily record data is carried out
Acquisition, can be realized by log acquisition server.In addition, being shown in the form of server in Fig. 1, characterization log collection can
To run on the server, certainly, which is only not single one, it is also possible to use cluster or distributed structure/architecture.
The Log Collect System of this specification embodiment poll Log Collect System different from the past.Specifically,
One, the present embodiment Log Collect System has higher collecting efficiency, because previous Log Collect System, is acquired using poll
Mode all journal files are all acquired, there are some files that new data is not written also to be acquired, reduce and adopt
Collect efficiency；Second, the present embodiment Log Collect System can effectively prevent daily record data to lose, because of previous log collection system
System is polled acquisition according to the specified period, and the filename of the former journal file after occurrence log rotation is changed, led to not
For the problem that the former journal file with new filename carries out log collection, there is daily record data loss, and the present embodiment system
System is then directed to the journal file after occurrence log rotates and is acquired；Third, the present embodiment is used to be carried out pair based on event type
The log collection answered can effectively reduce the difficulty of log collection without the concern for daily record rotation rule and daily record Rename rule
Degree simplifies log collection rule.
It should be noted that log collection described in this specification embodiment is realized based on linux system, utilize
System provide inotify mechanism come monitor journal file change event, addition inotify monitoring Log Directory or daily record
File can all send event notice when establishment, deletion, movement, modification occurs.This programme mainly uses three things therein
Part：(file moves by IN_MODIFY (file modification event), IN_MOVED_FROM (file removal event) and IN_MOVED_TO
Event).If the above three event for the same old file name occurs simultaneously based on same catalogue, it is believed that daily record text
Part occurrence log rotates.The type for the event being specifically applied to can be selected according to actual demand, be implemented in this specification
It is illustrative only in example, does not constitute the limitation to technical scheme.
Based on above-mentioned scene, the scheme of this specification is described in detail below.
Fig. 2 is the flow diagram of a kind of log collection method that this specification embodiment provides, and this method specifically can be with
Include the following steps：
Step S202：The file for obtaining journal file changes event.
Journal file is recorded on the event occurred in operating system or other software operation or the difference in communication software
The file of message between user.The embodiment of the present application is illustrated by taking operating system generation event as an example, day mentioned here
Will file changes event：The events such as document creation, removal, immigration, the modification of the journal file of system, all can be by being
Transmission event of uniting notice.Event is changed by file, it can be realized that file changes front and back correlation log fileinfo, so as to
The acquisition of daily record data is carried out for specified journal file.
Step S204：The type that event is changed according to the file, records the corresponding fileinfo of the journal file,
In, the fileinfo includes：The journal file changes front and back fileinfo and changes label information.
After receiving file variation event, it is necessary first to judge file change event type, for concrete type into
The processing of the corresponding event of row, and record fileinfo corresponding with the journal file.It is repaiied for file for example, file changes event
Change event (IN_MODIFY), determines that the corresponding journal file of the event is after needing the journal file that is acquired, to record this
The new filename of journal file and corresponding document misregistration amount.
It should be noted that is stored in the corresponding journal file of new filename mentioned here is the number of original daily record
According to.Specifically, during daily record rotates, renaming can be carried out to former journal file, then generate one it is empty, named
For old file name new journal file be used for record after daily record data.For example, the original of the former journal file of system database
The entitled dal-digest.log of file needs to carry out renamed as dal- to former journal file when carrying out daily record rotation
Then digest.log.1 generates a new journal file that is empty, being named as dal-digest.log, for recording rotation
Newly generated daily record data afterwards.
Step S206：Based on the fileinfo, the data of the journal file are acquired.
Document misregistration amount mentioned here be for journal file is marked, can be accurate when to acquire next time
It navigates to and continues log data acquisition since where current log file.
For example, the old file name of the former journal file of system database is dal-digest.log, it is assumed that rotated in daily record
Before, last time collects the position of the 90th row in former journal file, and the label 222 of document misregistration amount is carried out to it；Carry out
After daily record rotation, renamed as dal-digest.log.1 is carried out to former journal file, while obtaining the entitled dal- of file
The new journal file of the blank of digest.log；After listening to daily record rotation event, trigger to dal-digest.log.1 days
The acquisition of will data proceeds by the acquisition of daily record data based on last document misregistration amount 222.In this way, will not both lose day
Will data will not acquire the data acquired again, can effectively improve the working efficiency of log collection.
According to above-described embodiment it can be appreciated that by way of being monitored to specified Log Directory, if monitoring
File has occurred for same journal file in same catalogue and removes event, file immigration event and file modification event, then
It is believed that journal file occurrence log rotation；Further, former journal file is determined according to new filename, further according to former daily record
Document misregistration amount in file carries out the acquisition of daily record data, may be implemented to be acquired for the journal file rotated,
When avoiding poll acquisition the problem of repeated acquisition, the efficiency of log collection can be improved；Can also occurrence log rotation after,
The acquisition for carrying out daily record data in time, can effectively prevent the loss of daily record data；By the way of being acquired based on event,
Without the concern for rotation rule and daily record Rename rule, the rule of log collection can be simplified, reduce the difficulty of log collection.
In this specification one or more embodiment, before the file variation event for obtaining journal file, may be used also
To include：According to preset monitoring period and preset Log Directory, the specified Log Directory is monitored.
In order to improve log collection efficiency, the Log Directory of setting monitoring is generally required.The Log Directory can according to
The actual demand at family carries out additions and deletions, for example, it is desired to be monitored to all journal files in some system database, it is assumed that be
System database produces new journal file, then needs the new journal file being added in Log Directory, to realize pair
The acquisition of the monitoring of all journal files and data in the system database.
It should be noted that the file variation event may include：File removes event (IN_MOVED_FROM), text
Part moves at least one of event (IN_MOVED_TO) and file modification event (IN_MODIFY) type.
File removes event (IN_MOVED_FROM) and removes monitored catalogue for monitoring journal file；File moves into thing
Part (IN_MOVED_TO) is moved into monitored catalogue for monitoring journal file；File modification event (IN_MODIFY) is for supervising
File is controlled by modification event, for example, being written into new daily record data or file name generation modification etc. in file.It is applying
In, the type of monitor event is defined according to event mask, and monitored target journaling is defined by filec descriptor and path
File.
In this specification one or more embodiment, the fileinfo before the journal file changes includes：
The variation label information of old file name, the journal file includes：cookie；If the type that the file changes event is file
Removal event then records the cookie and old file name of the corresponding journal file.
Include the information such as cookie and old file name in the list of thing of file removal event mentioned here.By above
It is described it is found that be the journal file of blank generated after rotation in the corresponding file of the old file name, it is newly generated for recording
Daily record data.Wherein, cookie here is used for being associated monitoring to two events.
In this specification one or more embodiment, the fileinfo after the journal file changes includes：
New filename；If the type that the file changes event, which is file, moves into event, judge this document move into the cookie of event with
Whether the cookie that the file removes event is identical；If so, record cookie, old file name and new filename.
From the foregoing it will be appreciated that cookie is used for recording associated two events, can be used for simultaneously to text herein
Part removes event and file moves into event and is monitored, to determine whether to be directed to the same journal file based on the same catalogue
Two events carried out, if (that is, file removes event and file moves into event cookie records having the same), then record
Cookie, old file name and the new filename of lower monitored journal file.
In this specification one or more embodiment, the variation label information of the journal file further includes：File
Offset；If the type that the file changes event is file modification event, judge the journal file file fingerprint whether
Variation；If so, record old file name and document misregistration amount；Wherein, after the document misregistration amount is used to acquire the last time
Journal file is marked.
It should be noted that each journal file has its unique file fingerprint, if journal file is changed
Or it is mobile, can all file fingerprint be caused to change.In practical applications, file fingerprint may include dev (device number) and
Inode (index node) can determine the specifying informations such as creation time and the modification time of the journal file based on inode.Cause
This, can judge whether the journal file rotates according to whether the file fingerprint of journal file changes.If taking turns
Turn (that is, this document fingerprint changes), then records corresponding old file name and document misregistration amount.
It is described to be based on the fileinfo in this specification one or more embodiment, acquire the journal file
Data, can specifically include：If file removes event, file moves into event and file modification event is directed to identical original
Name carries out data acquisition then according to the document misregistration amount to the corresponding journal file of the new filename.
If for same former journal file be carried out at the same time that file removes event, file moves into event in same catalogue and
File modification event can then determine that the journal file rotates.
For example, if the journal file acquired, first documents fingerprint is thought when detecting file fingerprint variation
The journal file rotates, then document misregistration amount when file rotates occurs for mark this document, recorded (old file name,
Document misregistration amount).Then setting current file offset is 0, and setting file fingerprint is dev (device number)+inode of new file
(index node), wherein store most of attribute of this journal file in inode, such as create, modification time.But
It is that filename and node number are not included in this inode node.This is primarily due to inode nodes and arranges in order, so
System kernel can use simple algorithm, so that it may to obtain inode node numbers.When detecting that same journal file is relevant
When tri- events of IN_MOVED_FROM, IN_MOVED_TO, IN_MODIFY are all recorded, that is, file removes event
(cookie, old file name), file move into (cookie, old file name, the new filename) of event and the (original of file modification event
Filename, document misregistration amount) when being all recorded, then the journal file after being rotated to the daily record is acquired processing：Use file
Corresponding log collection is carried out in file after offset to renaming, ensures the daily record data for acquisition caused by file rotation
It does not lose, after log collection, deletes this daily record and rotate relevant record.
In this specification one or more embodiment, after the data of the acquisition journal file, further include：
The document misregistration amount and file fingerprint of the journal file after acquisition are set.
The log collection of the technical solution of the present embodiment is the process of a persistent loop, is illustrated in figure 3 this specification
The method schematic diagram for the log collection based on document misregistration amount that embodiment provides, step S302：Obtain the file of journal file
Variation event；Step S304：The type that event is changed according to the file, records the corresponding fileinfo of the journal file,
Wherein, the fileinfo includes：At least one of new filename and document misregistration amount after the journal file variation；Step
Rapid S306：Based on the new filename and the document misregistration amount, the data of the journal file are acquired, journal file is completed and adopts
After collection, step S308：The document misregistration amount and file fingerprint of the journal file after acquisition are set, for example, it is desired to having adopted
The offset zero setting for collecting the journal file completed, is marked to not acquiring the document misregistration amount of journal file of completion.
Based on same thinking, this illustrates that embodiment also provides a kind of computer-readable medium, and the media storage has meter
Calculation machine readable instruction, the computer-readable instruction can be executed by processor to realize the side described in any embodiment as above
Method.
For the ease of understanding technical scheme, a specific embodiment is provided below, is illustrated in figure 4 this explanation
The flow diagram for a kind of the daily record monitoring and gatherer process that book provides, it is assumed that the journal file of application system is service-
Digest.log, when there is daily record rotation across day, service-digest.log is renamed into service-
Then digest.log.2017-11-08 creates an empty service-digest.log and is used for subsequent log recording.Day
Will capture program can carry out the record of following event：
S401：Definition monitoring Log Directory；
S402：It obtains file and changes event；
S403：Determine event type；Further, according to event type, corresponding step is executed, it is specific as follows：
S404：IN_MOVED_FROM：Record (cookie：111, filename：Service-digest.log),
S405：IN_MOVED_TO：(cookie：111, filename：Service-digest.log.2017-11-08), lead to
The cookie matched and searcheds with the actual records of IN_MOVED_FROM are crossed, if finding identical cookie records, it is known that
Service-digest.log files are renamed into service-digest.log.2017-11-08, record (cookie：
111, old file name：Service-digest.log, filename：service-digest.log.2017-11-08).
S406：IN_MODIFY：(filename：Service-digest.log), judge whether to have occurred by file fingerprint
Daily record rotates, if so, record (filename：Service-digest.log, original offset：222).
S407：Record (the cookie for finally combining step S405 to generate：111, old file name：service-
Digest.log, filename：) and the record (original that generates of step S406 service-digest.log.2017-11-08
Name：Service-digest.log, original offset：222) this two records carry out daily record round-robin processing：To service-
The daily record data after acquisition offset 222 is removed in digest.log.2017-11-08 files.
Based on same thinking, this specification embodiment also provides a kind of log acquisition apparatus, is illustrated in figure 5 this explanation
A kind of structural schematic diagram for log acquisition apparatus that book embodiment provides, the device can specifically include：
Acquisition module 501, the file for obtaining journal file change event；
Logging modle 502 changes the type of event according to the file, records the file letter of the corresponding journal file
Breath, wherein the fileinfo includes：The journal file changes front and back fileinfo and changes label information；
Acquisition module 503 is based on the fileinfo, acquires the data of the journal file.
Further, further include monitoring module 504, before the file variation event for obtaining journal file, the monitoring
Module 504 is monitored the specified Log Directory according to preset monitoring period and preset Log Directory.
Further, the file variation event includes：File removes event, file moves into event and file modification event
At least one of type.
Further, the fileinfo before the journal file variation includes：Old file name, the journal file
Changing label information includes：cookie；
If the type that the file changes event, which is file, removes event, the corresponding journal file is recorded
Cookie and old file name.
Further, the fileinfo after the journal file variation includes：New filename；
If the type that the file changes event, which is file, moves into event, judge that this document moves into cookie and the institute of event
Whether the cookie for stating file removal event is identical；
If so, record cookie, old file name and new filename.
Further, the variation label information of the journal file further includes：Document misregistration amount；
If the type that the file changes event is file modification event, judge the journal file file fingerprint whether
Variation；
If so, record old file name and document misregistration amount；
Wherein, the document misregistration amount is used to that the journal file after last acquire to be marked.
Further, described to be based on the fileinfo, the data of the journal file are acquired, are specifically included：
If file removes event, file moves into event and file modification event is directed to identical old file name, according to institute
Document misregistration amount is stated, the journal file corresponding to the new filename carries out data acquisition.
Further, after the data of the acquisition journal file, further include：
The document misregistration amount and file fingerprint of the journal file after acquisition are set.
Based on same thinking, this specification embodiment also provides a kind of electronic equipment, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by described at least one
A processor executes, so that at least one processor can：
The file for obtaining journal file changes event；
The type that event is changed according to the file, records the corresponding fileinfo of the journal file, wherein the text
Part information includes：The journal file changes front and back fileinfo and changes label information；
Based on the fileinfo, the data of the journal file are acquired.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the action recorded in detail in the claims or step can be come according to different from the sequence in embodiment
It executes and desired result still may be implemented.In addition, the process described in the accompanying drawings not necessarily require show it is specific suitable
Sequence or consecutive order could realize desired result.In some embodiments, multitasking and parallel processing be also can
With or it may be advantageous.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For electronic equipment, nonvolatile computer storage media embodiment, since it is substantially similar to the method embodiment, so description
It is fairly simple, the relevent part can refer to the partial explaination of embodiments of method.
Device that this specification embodiment provides, electronic equipment, nonvolatile computer storage media with method are corresponding
, therefore, device, electronic equipment, nonvolatile computer storage media also there is the Advantageous similar with corresponding method to imitate
Fruit, since the advantageous effects of method being described in detail above, which is not described herein again corresponding intrument,
The advantageous effects of electronic equipment, nonvolatile computer storage media.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " patrols
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed are most generally used at present
Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also answer
This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,
The hardware circuit for realizing the logical method flow can be readily available.
Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing
The computer for the computer readable program code (such as software or firmware) that device and storage can be executed by (micro-) processor can
Read medium, logic gate, switch, application-specific integrated circuit (Application Specific Integrated Circuit,
ASIC), the form of programmable logic controller (PLC) and embedded microcontroller, the example of controller includes but not limited to following microcontroller
Device：ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, are deposited
Memory controller is also implemented as a part for the control logic of memory.It is also known in the art that in addition to
Pure computer readable program code mode is realized other than controller, can be made completely by the way that method and step is carried out programming in logic
Controller is obtained in the form of logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc. to come in fact
Existing identical function.Therefore this controller is considered a kind of hardware component, and to including for realizing various in it
The device of function can also be considered as the structure in hardware component.Or even, it can will be regarded for realizing the device of various functions
For either the software module of implementation method can be the structure in hardware component again.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
For convenience of description, it is divided into various units when description apparatus above with function to describe respectively.Certainly, implementing this
The function of each unit is realized can in the same or multiple software and or hardware when specification one or more embodiment.
It should be understood by those skilled in the art that, this specification embodiment can be provided as method, system or computer program
Product.Therefore, this specification embodiment can be used complete hardware embodiment, complete software embodiment or combine software and hardware
The form of the embodiment of aspect.Moreover, it wherein includes that computer is available that this specification embodiment, which can be used in one or more,
It is real in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form for the computer program product applied.
This specification is with reference to the method, equipment (system) and computer program product according to this specification embodiment
Flowchart and/or the block diagram describes.It should be understood that can be realized by computer program instructions every in flowchart and/or the block diagram
The combination of flow and/or box in one flow and/or box and flowchart and/or the block diagram.These computers can be provided
Processor of the program instruction to all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices
To generate a machine so that the instruction executed by computer or the processor of other programmable data processing devices generates use
In the dress for realizing the function of being specified in one flow of flow chart or multiple flows and/or one box of block diagram or multiple boxes
It sets.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology realizes information storage.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic tape cassette, tape magnetic disk storage or other magnetic storage apparatus
Or any other non-transmission medium, it can be used for storage and can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
Including so that process, method, commodity or equipment including a series of elements include not only those elements, but also wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that wanted including described
There is also other identical elements in the process of element, method, commodity or equipment.
This specification can describe in the general context of computer-executable instructions executed by a computer, such as journey
Sequence module.Usually, program module include routines performing specific tasks or implementing specific abstract data types, program, object,
Component, data structure etc..Specification can also be put into practice in a distributed computing environment, in these distributed computing environments,
By executing task by the connected remote processing devices of communication network.In a distributed computing environment, program module can
With in the local and remote computer storage media including storage device.
Each embodiment in this specification is described in a progressive manner, identical similar portion between each embodiment
Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so description is fairly simple, related place is referring to embodiment of the method
Part explanation.
The foregoing is merely this specification embodiments, are not intended to limit this application.For those skilled in the art
For, the application can have various modifications and variations.It is all within spirit herein and principle made by any modification, equivalent
Replace, improve etc., it should be included within the scope of claims hereof.
Claims (17)
1. a kind of log collection method, including：
The file for obtaining journal file changes event；
The type that event is changed according to the file, records the corresponding fileinfo of the journal file, wherein the file letter
Breath includes：The journal file changes front and back fileinfo and changes label information；
Based on the fileinfo, the data of the journal file are acquired.
2. the method as described in claim 1 further includes before the file variation event for obtaining journal file：
According to preset monitoring period and preset Log Directory, the specified Log Directory is monitored.
3. method as claimed in claim 2, the file changes event and includes：File remove event, file move into event and
At least one of file modification event type.
4. method as claimed in claim 3, the fileinfo before the journal file changes include：Old file name, the day
The variation label information of will file includes：cookie；
If the type that the file changes event, which is file, removes event, record the corresponding journal file cookie and
Old file name.
5. method as claimed in claim 4, the fileinfo after the journal file changes include：New filename；
If the type that the file changes event, which is file, moves into event, judge that this document moves into the cookie of event and the text
Whether the cookie that part removes event is identical；
If so, record cookie, old file name and new filename.
6. the variation label information of method as claimed in claim 5, the journal file further includes：Document misregistration amount；
If the type that the file changes event is file modification event, judge whether the file fingerprint of the journal file becomes
Change；
If so, record old file name and document misregistration amount；
Wherein, the document misregistration amount is used to that the journal file after last acquire to be marked.
7. method as claimed in claim 6, described be based on the fileinfo, the data of the journal file are acquired, specifically
Including：
If file removes event, file moves into event and file modification event is directed to identical old file name, according to the text
Part offset, the journal file corresponding to the new filename carry out data acquisition.
8. the method as described in claim 1, after the data of the acquisition journal file, further include：
The document misregistration amount and file fingerprint of the journal file after acquisition are set.
9. a kind of log acquisition apparatus, including：
Acquisition module, the file for obtaining journal file change event；
Logging modle changes the type of event according to the file, records the fileinfo of the corresponding journal file,
In, the fileinfo includes：The journal file changes front and back fileinfo and changes label information；
Acquisition module is based on the fileinfo, acquires the data of the journal file.
10. device as claimed in claim 9 further includes before the file variation event for obtaining journal file：
According to preset monitoring period and preset Log Directory, the specified Log Directory is monitored.
11. device as claimed in claim 10, the file changes event and includes：File removes event, file moves into event
With at least one of file modification event type.
12. device as claimed in claim 11, the fileinfo before the journal file changes include：Old file name,
The variation label information of the journal file includes：cookie；
If the type that the file changes event, which is file, removes event, record the corresponding journal file cookie and
Old file name.
13. device as claimed in claim 12, the fileinfo after the journal file changes include：New filename；
If the type that the file changes event, which is file, moves into event, judge that this document moves into the cookie of event and the text
Whether the cookie that part removes event is identical；
If so, record cookie, old file name and new filename.
14. the variation label information of device as claimed in claim 13, the journal file further includes：Document misregistration amount；
If the type that the file changes event is file modification event, judge whether the file fingerprint of the journal file becomes
Change；
If so, record old file name and document misregistration amount；
Wherein, the document misregistration amount is used to that the journal file after last acquire to be marked.
15. device as claimed in claim 14, described to be based on the fileinfo, the data of the journal file are acquired, are had
Body includes：
If file removes event, file moves into event and file modification event is directed to identical old file name, according to the text
Part offset, the journal file corresponding to the new filename carry out data acquisition.
16. device as claimed in claim 9, after the data of the acquisition journal file, further include：
The document misregistration amount and file fingerprint of the journal file after acquisition are set.
17. a kind of electronic equipment, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by least one place
It manages device to execute, so that at least one processor can：
The file for obtaining journal file changes event；
The type that event is changed according to the file, records the corresponding fileinfo of the journal file, wherein the file letter
Breath includes：The journal file changes front and back fileinfo and changes label information；
Based on the fileinfo, the data of the journal file are acquired.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810285017.4A CN108733546A (en) | 2018-04-02 | 2018-04-02 | A kind of log collection method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810285017.4A CN108733546A (en) | 2018-04-02 | 2018-04-02 | A kind of log collection method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108733546A true CN108733546A (en) | 2018-11-02 |
Family
ID=63941122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810285017.4A Pending CN108733546A (en) | 2018-04-02 | 2018-04-02 | A kind of log collection method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108733546A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032496A (en) * | 2019-04-19 | 2019-07-19 | 杭州玳数科技有限公司 | A kind of log collection method and system for supporting diversified log merging |
| CN111625397A (en) * | 2020-04-14 | 2020-09-04 | 北京捷通华声科技股份有限公司 | Service log backup method, cluster, device, electronic equipment and storage medium |
| CN111856249A (en) * | 2020-07-24 | 2020-10-30 | 安测半导体技术(江苏)有限公司 | Chip test monitoring method, client and system |
| CN113032183A (en) * | 2021-03-24 | 2021-06-25 | 西安闻泰信息技术有限公司 | System management method, device, computer equipment and storage medium |
| CN113138967A (en) * | 2021-05-08 | 2021-07-20 | 贵州全安密灵科技有限公司 | Data information acquisition method, black box, detonator and storage medium |
| CN113760683A (en) * | 2021-08-31 | 2021-12-07 | 银清科技有限公司 | Log collection method and device |
| CN114968962A (en) * | 2022-08-01 | 2022-08-30 | 中瓴智行(成都)科技有限公司 | Log management method and system based on multi-core communication and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100161565A1 (en) * | 2008-12-18 | 2010-06-24 | Electronics And Telecommunications Research Institute | Cluster data management system and method for data restoration using shared redo log in cluster data management system |
| CN102571476A (en) * | 2010-12-27 | 2012-07-11 | ***股份有限公司 | Method and device for monitoring terminal command line in real time |
| CN102622267A (en) * | 2012-03-14 | 2012-08-01 | 深圳市紫金支点技术股份有限公司 | Log acquisition method and system |
| CN103412893A (en) * | 2013-07-24 | 2013-11-27 | 广东电子工业研究院有限公司 | Collecting system and collecting method of logs |
| CN104714878A (en) * | 2013-12-11 | 2015-06-17 | 阿里巴巴集团控股有限公司 | Method and device for collecting log data |
| CN106294351A (en) * | 2015-05-13 | 2017-01-04 | 阿里巴巴集团控股有限公司 | Log event treating method and apparatus |
| CN106569936A (en) * | 2016-09-26 | 2017-04-19 | 深圳盒子支付信息技术有限公司 | Method and system for acquiring scrolling log in real time |
| WO2017072840A1 (en) * | 2015-10-26 | 2017-05-04 | 株式会社日立製作所 | Log collection system and log collection method |
-
2018
- 2018-04-02 CN CN201810285017.4A patent/CN108733546A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100161565A1 (en) * | 2008-12-18 | 2010-06-24 | Electronics And Telecommunications Research Institute | Cluster data management system and method for data restoration using shared redo log in cluster data management system |
| CN102571476A (en) * | 2010-12-27 | 2012-07-11 | ***股份有限公司 | Method and device for monitoring terminal command line in real time |
| CN102622267A (en) * | 2012-03-14 | 2012-08-01 | 深圳市紫金支点技术股份有限公司 | Log acquisition method and system |
| CN103412893A (en) * | 2013-07-24 | 2013-11-27 | 广东电子工业研究院有限公司 | Collecting system and collecting method of logs |
| CN104714878A (en) * | 2013-12-11 | 2015-06-17 | 阿里巴巴集团控股有限公司 | Method and device for collecting log data |
| CN106294351A (en) * | 2015-05-13 | 2017-01-04 | 阿里巴巴集团控股有限公司 | Log event treating method and apparatus |
| WO2017072840A1 (en) * | 2015-10-26 | 2017-05-04 | 株式会社日立製作所 | Log collection system and log collection method |
| CN106569936A (en) * | 2016-09-26 | 2017-04-19 | 深圳盒子支付信息技术有限公司 | Method and system for acquiring scrolling log in real time |
Non-Patent Citations (1)
| Title |
|---|
| 鲍一丹: "基于Windows平台的日志提取与分析", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032496A (en) * | 2019-04-19 | 2019-07-19 | 杭州玳数科技有限公司 | A kind of log collection method and system for supporting diversified log merging |
| CN110032496B (en) * | 2019-04-19 | 2023-10-13 | 杭州玳数科技有限公司 | Log acquisition method and system supporting diversified log merging |
| CN111625397A (en) * | 2020-04-14 | 2020-09-04 | 北京捷通华声科技股份有限公司 | Service log backup method, cluster, device, electronic equipment and storage medium |
| CN111625397B (en) * | 2020-04-14 | 2023-09-12 | 北京捷通华声科技股份有限公司 | Service log backup method, cluster, device, electronic equipment and storage medium |
| CN111856249A (en) * | 2020-07-24 | 2020-10-30 | 安测半导体技术(江苏)有限公司 | Chip test monitoring method, client and system |
| CN113032183A (en) * | 2021-03-24 | 2021-06-25 | 西安闻泰信息技术有限公司 | System management method, device, computer equipment and storage medium |
| CN113138967A (en) * | 2021-05-08 | 2021-07-20 | 贵州全安密灵科技有限公司 | Data information acquisition method, black box, detonator and storage medium |
| CN113760683A (en) * | 2021-08-31 | 2021-12-07 | 银清科技有限公司 | Log collection method and device |
| CN113760683B (en) * | 2021-08-31 | 2024-02-20 | 银清科技有限公司 | Log acquisition method and device |
| CN114968962A (en) * | 2022-08-01 | 2022-08-30 | 中瓴智行(成都)科技有限公司 | Log management method and system based on multi-core communication and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108733546A (en) | A kind of log collection method, device and equipment | |
| CN106021017B (en) | For backing up and restoring the method and system of clone file | |
| CN110007957A (en) | Call link construction method, device and equipment | |
| CN107526667A (en) | A kind of Indexes Abnormality detection method, device and electronic equipment | |
| CN108171267A (en) | User group partitioning method and device, information push method and device | |
| CN108345977A (en) | A kind of method for processing business and device | |
| CN108959341A (en) | A kind of method, device and equipment that data are synchronous | |
| CN107391101A (en) | A kind of information processing method and device | |
| CN110807013B (en) | Data migration method and device for distributed data storage cluster | |
| CN109299096A (en) | A kind of processing method of pipelined data, device and equipment | |
| CN111078723B (en) | Data processing method and device for block chain browser | |
| CN107577697A (en) | A kind of data processing method, device and equipment | |
| EP3640813A1 (en) | Random walk method, apparatus and device, and cluster-based random walk method, apparatus and device | |
| CN108243032A (en) | A kind of acquisition methods of class of service information, device and equipment | |
| CN109766167A (en) | Method, apparatus, system and the equipment of timed task distribution | |
| CN109284066A (en) | A kind of data processing method, device, equipment and system | |
| CN109597678A (en) | Task processing method and device | |
| CN108334340A (en) | A kind of Cross-environment data communications method, device and system | |
| CN105868216A (en) | Method for realizing object expiration operation and device and equipment | |
| CN110245137B (en) | Index processing method, device and equipment | |
| CN109739814A (en) | A kind of storage of file and method for down loading, device and equipment | |
| CN108390914A (en) | A kind of service update method and device, system | |
| CN108304455A (en) | A kind of method, apparatus and equipment of processing business request | |
| CN110134735A (en) | The storage method and device of distributed transaction log | |
| CN116048977B (en) | Test method and device based on data reduction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201021 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20201021 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |
|
| RJ01 | Rejection of invention patent application after publication |