CN104714878B - A kind of method and device of collector journal data - Google Patents
A kind of method and device of collector journal data Download PDFInfo
- Publication number
- CN104714878B CN104714878B CN201310676525.2A CN201310676525A CN104714878B CN 104714878 B CN104714878 B CN 104714878B CN 201310676525 A CN201310676525 A CN 201310676525A CN 104714878 B CN104714878 B CN 104714878B
- Authority
- CN
- China
- Prior art keywords
- file
- specified
- journal
- journal file
- daily record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of method and device of collector journal data, belong to field of computer technology.Methods described includes：The JFS specified by the monitoring of file change notification interface；Obtain the event for specifying journal file to be written into daily record data；Judge the current file fingerprint of the specified journal file, it is whether consistent with the file fingerprint before the specified journal file；If it is inconsistent, acquisition and new file name, old file name and original offset according to corresponding to the specified journal file, the collector journal data from journal file corresponding to journal file corresponding to the old file name and the new file name.Described device includes：Monitoring modular, the first acquisition module, the first judge module and the first collection module.The present invention can be collected into the daily record data being recorded in the journal file after renaming so that can be collected into complete complete daily record data.
Description
Technical field
The application is related to field of computer technology, and in particular to a kind of method and device of collector journal data.
Background technology
With the development of computer technology, the function of computer is also more and more, and computer turns into essential work
Tool.Computer needs to run by program, for the ease of being safeguarded to program, ensures computer normal operation, usually
By the logdata record such as caused running status in program operation process in journal file is specified, pass through query log files
It can be obtained by the related information of program.In a computer cluster, generally require some in each computer to preset journey
The collection of log data of sequence carries out unified supervision and processing on server to daily record data.
The method of existing collector journal data is as follows：According to the filename of specified journal file, specified daily record text is inquired
Part, the collector journal data from specified journal file.
However, in order to prevent that journal file is excessive, can be according to preset rules（Time interval or file size etc.）Carry out day
Will rollback, i.e.,：When specified journal file meets preset rules, specified journal file is renamed（It is revised as other texts
Part name）, then newly-built one as the old file name of specified journal file blank journal file record rollback after day
Will data.The method of existing collector journal data, it is according to the filename for specifying journal file, is collected from specified journal file
Daily record data, after log rollback, the daily record data that is recorded in the journal file after renaming just collect less than so that receive
The daily record data of collection is not complete.
The content of the invention
Technical problems to be solved in this application are to provide a kind of method and device of collector journal data, can collected
To the daily record data being recorded in the journal file after renaming so that complete complete daily record data can be collected into.
In order to solve the above problems, this application discloses a kind of method of collector journal information, methods described includes：
The JFS specified by the monitoring of file change notification interface；
Obtain the event for specifying journal file to be written into daily record data；
Judge the current file fingerprint of the specified journal file, be with the file fingerprint before the specified journal file
It is no consistent；
If it is inconsistent, acquisition and new file name, old file name and original text according to corresponding to the specified journal file
Part offset, the collector journal number from journal file corresponding to journal file corresponding to the old file name and the new file name
According to.
Further, methods described also includes：
After the JFS specified by the monitoring of file change notification interface, in addition to：
Obtain the event that the specified journal file is renamed；
Record new file name and old file name corresponding to the specified journal file.
Further, the file change notification interface includes：In inotify in linux system, windows system
File system monitoring FileSystemWatcher；
Correspondingly, the event that the specified journal file is written into daily record data includes：File in inotify has new
In_modify events are write, the change Changed events in FileSystemWatcher；
Correspondingly, the event that the specified journal file is renamed includes：File in inotify removes in_move_
From events and file move into in_move_to events, the renaming Renamed events in FileSystemWatcher.
Further, the acquisition and new file name, old file name and original text according to corresponding to the specified journal file
Part offset, the collector journal number from journal file corresponding to journal file corresponding to the old file name and the new file name
According to, including：
Using the current document misregistration amount of the specified journal file and filename as the original offset and
The old file name is recorded；
The current document misregistration amount of the specified journal file is reset to 0, collects the day in the specified journal file
Will data, regenerate and record the file fingerprint of the specified journal file；
When judging to there are the record of new file name and old file name corresponding to the specified journal file, according to described
Original offset, the collector journal data from journal file corresponding to the new file name.
Further, described obtain specifies journal file to be written into after the event of daily record data, judges the specified day
Before whether file fingerprint before the current file fingerprint of will file, with the specified journal file is consistent, in addition to：
Judge whether the specified journal file is collected daily record data for the first time；
Collected if it is, the specified journal file is added in file set, by the text of the specified journal file
Part offset is arranged to 0, collects the daily record data in the specified journal file, generates and record the specified journal file
File fingerprint；
If it is not, then the file fingerprint for judging that the specified journal file is current is performed, with the specified daily record
The whether consistent step of file fingerprint before file.
Further, the file fingerprint for judging that the specified journal file is current, with the specified journal file it
After whether preceding file fingerprint is consistent, in addition to：
If consistent, according to the current document misregistration amount of the specified journal file, from the specified journal file
Collector journal data, and reset after collector journal data the document misregistration amount of the specified journal file.
In order to solve the above problems, this application discloses a kind of device of collector journal data, described device includes：
Monitoring modular, for the JFS specified by the monitoring of file change notification interface；
First acquisition module, the event of daily record data is written into for obtaining specified journal file；
First judge module, the file fingerprint current for judging the specified journal file, with the specified daily record text
Whether the file fingerprint before part is consistent；
First collection module, if the judged result for first judge module is inconsistent, obtain simultaneously basis
New file name, old file name and original offset corresponding to the specified journal file, from day corresponding to the old file name
Collector journal data in journal file corresponding to will file and the new file name.
Further, described device also includes：
Second acquisition module, the event being renamed for obtaining the specified journal file；
Logging modle, for recording new file name and old file name corresponding to the specified journal file.
Further, the file change notification interface includes：In inotify in linux system, windows system
File system monitoring FileSystemWatcher；
Correspondingly, the event that the specified journal file is written into daily record data includes：File in inotify has new
In_modify events are write, the change Changed events in FileSystemWatcher；
Correspondingly, the event that the specified journal file is renamed includes：File in inotify removes in_move_
From events and file move into in_move_to events, the renaming Renamed events in FileSystemWatcher.
Further, first collection module includes：
Recording unit, if for inconsistent, by the specified journal file current document misregistration amount and file status
Not Zuo Wei the original offset and the old file name record；
Processing unit, for the current document misregistration amount of the specified journal file to be reset into 0, collect the specified day
Daily record data in will file, regenerate and record the file fingerprint of the specified journal file；
Collector unit, for there are new file name and the note of old file name corresponding to the specified journal file when judgement
During record, according to the original offset, the collector journal data from journal file corresponding to the new file name.
Further, described device also includes：
Second judge module, for obtaining after specified journal file is written into the event of daily record data, judge the finger
Determine the current file fingerprint of journal file, with the file fingerprint before the specified journal file it is whether consistent before, judge institute
State whether specified journal file is collected daily record data for the first time；
Processing module, if the judged result for second judge module is, the specified journal file is added
Enter to collecting in file set, the document misregistration amount of the specified journal file is arranged to 0, collects the specified journal file
In daily record data, generate and record the file fingerprint of the specified journal file；
Notification module, if the judged result for second judge module is to notify described first to judge
Module performs the file fingerprint for judging that the specified journal file is current, refers to the file before the specified journal file
The whether consistent step of line.
Further, described device also includes：
Second collection module, if the judged result for first judge module is consistent, specified according to described
The current document misregistration amount of journal file, the collector journal data from the specified journal file, and after collector journal data
Reset the document misregistration amount of the specified journal file.
Compared with prior art, the application can be obtained including following technique effect：
The JFS specified by the monitoring of file change notification interface, is written into when getting specified journal file
The event of daily record data, judge the file fingerprint for specifying journal file current, the file with the specified journal file of local record
Fingerprint is inconsistent（Prove that there occurs log rollback）When, obtain and new file name, original according to corresponding to specified journal file
Name and original offset, the collector journal number from journal file corresponding to journal file corresponding to old file name and new file name
According to so that the daily record data being recorded in the journal file after renaming can be collected into so that can be collected into complete neat
Full daily record data.
Certainly, implementing any product of the application must be not necessarily required to reach all the above technique effect simultaneously.
Brief description of the drawings
Accompanying drawing described herein is used for providing further understanding of the present application, forms the part of the application, this Shen
Schematic description and description please is used to explain the application, does not form the improper restriction to the application.In the accompanying drawings：
Fig. 1 is the method flow diagram of the first collector journal data of the embodiment of the present application；
Fig. 2 is the method flow diagram of second of collector journal data of the embodiment of the present application；
Fig. 3 is the apparatus structure schematic diagram of the first collector journal data of the embodiment of the present application；
Fig. 4 is the apparatus structure schematic diagram of second of collector journal data of the embodiment of the present application；
Fig. 5 is the apparatus structure schematic diagram of the third collector journal data of the embodiment of the present application；
Fig. 6 is the apparatus structure schematic diagram of the 4th kind of collector journal data of the embodiment of the present application.
Embodiment
Presently filed embodiment is described in detail below in conjunction with drawings and Examples, and thereby how the application is applied
Technological means can fully understand and implement according to this to solve technical problem and reach the implementation process of technical effect.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and internal memory.
Internal memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM).Internal memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moved
State random access memory (DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic cassette tape, the storage of tape magnetic rigid disk or other magnetic storage apparatus
Or any other non-transmission medium, the information that can be accessed by a computing device available for storage.Define, calculate according to herein
Machine computer-readable recording medium does not include non-temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
Embodiment describes
It is described further below with realization of the embodiment to the application method.As shown in figure 1, it is the embodiment of the present application
A kind of collector journal data method flow diagram, this method includes：
S101:The JFS specified by the monitoring of file change notification interface.
S102:Obtain the event for specifying journal file to be written into daily record data.
S103:Judge to specify the current file fingerprint of journal file, with the file fingerprint before specified journal file whether
Unanimously.
S104:If it is inconsistent, acquisition and new file name, old file name and original text according to corresponding to specified journal file
Part offset, the collector journal data from journal file corresponding to journal file corresponding to old file name and new file name.
Preferably, after monitoring the JFS specified by file change notification interface, in addition to：
Obtain the event for specifying journal file to be renamed；
Record specifies new file name and old file name corresponding to journal file.
Preferably, file change notification interface includes：File in inotify in linux system, windows system
System monitoring FileSystemWatcher；
Correspondingly, specified journal file is written into the event of daily record data and included：File in inotify has new write-in
In_modify events, the change Changed events in FileSystemWatcher；
Correspondingly, the event that specified journal file is renamed includes：File in inotify removes in_move_from
Event and file move into in_move_to events, the renaming Renamed events in FileSystemWatcher.
Preferably, acquisition and new file name, old file name and original offset according to corresponding to specified journal file, from
Collector journal data in journal file corresponding to journal file corresponding to old file name and new file name, including：
Using the current document misregistration amount of specified journal file and filename as original offset and old file name
Record；
The current document misregistration amount of specified journal file is reset to 0, collects the daily record data specified in journal file, weight
File fingerprint that is newly-generated and recording specified journal file；
It is inclined according to original when judging to there are the record of new file name and old file name corresponding to specified journal file
Shifting amount, the collector journal data from journal file corresponding to new file name.
Preferably, obtaining specifies journal file to be written into after the event of daily record data, judges to specify journal file current
File fingerprint, with the file fingerprint before specified journal file it is whether consistent before, in addition to：
Judge to specify whether journal file is collected daily record data for the first time；
Collected if it is, specified journal file is added in file set, by the document misregistration amount of specified journal file
0 is arranged to, the daily record data specified in journal file is collected, generates and record the file fingerprint of specified journal file；
If it is not, then the file fingerprint for judging to specify journal file current is performed, with the text before specified journal file
The whether consistent step of part fingerprint.
Preferably, judge the file fingerprint for specifying journal file current, be with the file fingerprint before specified journal file
It is no it is consistent after, in addition to：
If consistent, according to the current document misregistration amount of specified journal file, the collector journal from specified journal file
Data, and reset after collector journal data the document misregistration amount of specified journal file.
The method of collector journal data described in the present embodiment, the daily record text specified by the monitoring of file change notification interface
Part system, when the event for getting specified journal file and being written into daily record data, the file for judging to specify journal file current refers to
Line, it is inconsistent with the file fingerprint of the specified journal file of local record（Prove that there occurs log rollback）When, obtain simultaneously basis
Specify new file name, old file name and original offset corresponding to journal file, from journal file corresponding to old file name and
Collector journal data in journal file corresponding to new file name so that the journal file being recorded in after renaming can be collected into
In daily record data so that complete complete daily record data can be collected into.
It is each that the method for collector journal data described in the embodiment of the present application can collect linux system, windows systems etc.
Daily record data in kind system, in order to make it easy to understand, being carried out below exemplified by collecting the daily record data in linux system further
Ground illustrates, as shown in Fig. 2 a kind of method flow diagram of collector journal data for the embodiment of the present application, this method include：
S201：The JFS specified by inotify monitorings.
Wherein, the object of inotify monitorings is catalogue.After monitoring catalogue changes, inotify can be produced correspondingly in real time
Sequence of events.
S202：Judge whether to get in_modify（File has new write-in）Event, in_move_from events（Text
Part removes）Or in_move_to（File moves into）Event, if getting in_modify events, perform S203；If obtain
To in_move_from events, then S211 is performed；If getting in_move_to events, S213 is performed.
S203：Judge to specify whether journal file is collected daily record data for the first time, if it is, S204 is performed, it is no
Then, S205 is performed.
S204：Specified journal file is added to and collected in file set, the document misregistration amount of specified journal file is set
0 is set to, the daily record data specified in journal file is collected, generates and record the file fingerprint of specified journal file, then terminate.
Specifically, the word that predetermined word joint number can be collected from specified journal file passes through file as file fingerprint
Whether the content that the specified journal file of fingerprint judgement includes.100 bytes are made before can such as being collected from specified journal file
For file fingerprint.
S205：Judge to specify the current file fingerprint of journal file, with the file fingerprint before specified journal file whether
Unanimously, if it is inconsistent, performing S206；Otherwise, S210 is performed.
Specifically, if it is inconsistent, explanation specifies journal file, there occurs log rollback.
S206：Mark specifies journal file that rollback occurs, by the current document misregistration amount of specified journal file and filename
Recorded respectively as original offset and old file name, obtain offset record（Old file name, original offset）.
Wherein, document misregistration amount represents that the daily record data before this document offset has all been collected.
Such as：When apache2 access.log occurrence log rollbacks, access.log is renamed into
Access.log.1, then it is newly-built go out an access.log log data, document misregistration amount current access.log
For 3033, then offset is recorded as：（Old file name：Access.log, original offset：3033）.
S207：The current document misregistration amount of specified journal file is reset to 0, collects the daily record number specified in journal file
According to regenerating and record the file fingerprint of specified journal file.
S208：The record of new file name and old file name corresponding to specified journal file is judged whether, if deposited
Having, then performing S209；Otherwise, terminate.
Specifically, if before the generation of in_modify events, first there occurs in_move_from events and in_move_
To events, then it can there are the record of new file name and old file name corresponding to specified journal file.As there are S215
In new file name record（cookie：123, old file name：Access.log, new file name：access.log.1).
S209：According to original offset, collector journal data, Ran Houjie from journal file corresponding to new file name
Beam.
Such as：Recorded according to offset（Old file name：Access.log, original offset：3033）In original
Offset 3033, recorded from new file name（cookie：123, old file name：Access.log, new file name：
Access.log.1 collector journal data in journal file corresponding to the new file name access.log.1 in).
S210：According to the current document misregistration amount of specified journal file, the collector journal data from specified journal file, and
The document misregistration amount of specified journal file is reset after collector journal data.Then terminate.
S211：Whether the file for judging to be moved out of needs the specified journal file of collector journal data, if it is, holding
Row S212；Otherwise, terminate.
S212：The current filename of the cookie of event and specified journal file is recorded as old file name,
Obtain old file name record（Cookie, old file name）, then terminate.
Wherein, cookie is the numbering for connecting two events, as the numbering of related two events.
Such as：When apache2 access.log occurrence log rollbacks, access.log is renamed into
Access.log.1, then it is newly-built go out an access.log log data, and corresponding cookie is 123, then this
When, old file name is recorded as：（cookie：123, old file name：access.log).
S213：Obtain and move into cookie corresponding to file.
S214：By cookie, judge that it is that movement comes under same catalogue to move into file, if it is, holding
Row S215；Otherwise, terminate.
Specifically, whether there is identical cookie in record that can be by looking into in_move_from events, to judge to move
It is that movement comes under same catalogue to enter file with the file being moved out of.
S215：The filename of file be will move into as specifying the new file name of journal file to record, obtain new file
Name record（Cookie, old file name, new file name）.
Such as：When apache2 access.log occurrence log rollbacks, access.log is renamed into
Access.log.1, then it is newly-built go out an access.log log data, and corresponding cookie is 123, then this
When, new file name is recorded as：（cookie：123, old file name：Access.log, new file name：access.log.1).
S216：Original offset and old file name record have been judged whether, if there is having, has then performed S209；
Otherwise, terminate.
Specifically, when such as performing the step, executed S206, that is, it there are offset record（Old file name, original are inclined
Shifting amount）, then S209 can be performed.
The method of collector journal data described in the present embodiment, the daily record text specified by the monitoring of file change notification interface
Part system, when the event for getting specified journal file and being written into daily record data, the file for judging to specify journal file current refers to
Line, it is inconsistent with the file fingerprint of the specified journal file of local record（Prove that there occurs log rollback）When, obtain simultaneously basis
Specify new file name, old file name and original offset corresponding to journal file, from journal file corresponding to old file name and
Collector journal data in journal file corresponding to new file name so that the journal file being recorded in after renaming can be collected into
In daily record data so that complete complete daily record data can be collected into.
As shown in figure 3, it is the structure drawing of device of the embodiment of the present application.A kind of device of collector journal data, including：
Monitoring modular 301, for the JFS specified by the monitoring of file change notification interface；
First acquisition module 302, the event of daily record data is written into for obtaining specified journal file；
First judge module 303, for the file fingerprint for judging to specify journal file current, before specified journal file
File fingerprint it is whether consistent；
First collection module 304, if the judged result for the first judge module 303 is inconsistent, obtain simultaneously root
According to new file name, old file name and original offset corresponding to specified journal file, from daily record corresponding to the old file name
Collector journal data in journal file corresponding to file and the new file name.
Preferably, also include referring to Fig. 4, the device：
Second acquisition module 305, the event being renamed for obtaining specified journal file；
Logging modle 306, for recording new file name and old file name corresponding to specified journal file.
Preferably, file change notification interface includes：File in inotify in linux system, windows system
System monitoring FileSystemWatcher；
Correspondingly, specified journal file is written into the event of daily record data and included：File in inotify has new write-in
In_modify events, the change Changed events in FileSystemWatcher；
Correspondingly, the event that specified journal file is renamed includes：File in inotify removes in_move_from
Event and file move into in_move_to events, the renaming Renamed events in FileSystemWatcher.
Preferably, the first collection module 304 includes：
Recording unit, if for inconsistent, the current document misregistration amount of specified journal file and filename are made respectively
Recorded for original offset and old file name；
Processing unit, for the current document misregistration amount of specified journal file to be reset into 0, collect and specify in journal file
Daily record data, regenerate and record the file fingerprint of specified journal file；
Collector unit, for there are the record of new file name and old file name corresponding to specified journal file when judgement
When, according to original offset, the collector journal data from journal file corresponding to the new file name.
Preferably, also include referring to Fig. 5, the device：
Second judge module 307, for obtaining after specified journal file is written into the event of daily record data, judge to specify
The current file fingerprint of journal file, with the file fingerprint before specified journal file it is whether consistent before, judge to specify daily record
Whether file is collected daily record data for the first time；
Processing module 308, if the judged result for the second judge module 307 is to add specified journal file
To collecting in file set, the document misregistration amount of specified journal file is arranged to 0, collects the daily record number specified in journal file
According to generating and record the file fingerprint of specified journal file；
Notification module 309, if the judged result for the second judge module 307 is to notify the first judgement mould
303 pieces perform it is described judge to specify the current file fingerprint of journal file, with the file fingerprint before specified journal file whether
Consistent step.
Preferably, also include referring to Fig. 6, the device：
Second collection module 310, if the judged result for the first judge module 303 is consistent, according to specified day
The current document misregistration amount of will file, the collector journal data from specified journal file, and set again after collector journal data
Put the document misregistration amount of specified journal file.
The device of collector journal data described in the present embodiment, the daily record text specified by the monitoring of file change notification interface
Part system, when the event for getting specified journal file and being written into daily record data, the file for judging to specify journal file current refers to
Line, it is inconsistent with the file fingerprint of the specified journal file of local record（Prove that there occurs log rollback）When, obtain simultaneously basis
Specify new file name, old file name and original offset corresponding to journal file, from journal file corresponding to old file name and
Collector journal data in journal file corresponding to new file name so that the journal file being recorded in after renaming can be collected into
In daily record data so that complete complete daily record data can be collected into.
Some preferred embodiments of the application have shown and described in described above, but as previously described, it should be understood that the application
Be not limited to form disclosed herein, be not to be taken as the exclusion to other embodiment, and available for various other combinations,
Modification and environment, and above-mentioned teaching or the technology or knowledge of association area can be passed through in the scope of the invention is set forth herein
It is modified., then all should be in this Shen and the change and change that those skilled in the art are carried out do not depart from spirit and scope
Please be in the protection domain of appended claims.
Claims (12)
- A kind of 1. method of collector journal data, it is characterised in that methods described includes：The JFS specified by the monitoring of file change notification interface；Obtain the event for specifying journal file to be written into daily record data；Judge the current file fingerprint of the specified journal file, with the file fingerprint before the specified journal file whether one Cause, the file fingerprint is determined according to the word for the predetermined word joint number collected from the specified journal file；If it is inconsistent, obtain and new file name, old file name and original are inclined according to corresponding to the specified journal file Shifting amount, the collector journal data from journal file corresponding to journal file corresponding to the old file name and the new file name.
- 2. the method as described in claim 1, it is characterised in that the daily record specified by the monitoring of file change notification interface After file system, in addition to：Obtain the event that the specified journal file is renamed；Record new file name and old file name corresponding to the specified journal file.
- 3. method as claimed in claim 2, it is characterised in that the file change notification interface includes：In linux system File system monitoring FileSystemWatcher in inotify, windows system；Correspondingly, the event that the specified journal file is written into daily record data includes：File in inotify has new write-in In_modify events, the change Changed events in FileSystemWatcher；Correspondingly, the event that the specified journal file is renamed includes：File in inotify removes in_move_from Event and file move into in_move_to events, the renaming Renamed events in FileSystemWatcher.
- 4. method as claimed in claim 2, it is characterised in that the acquisition is simultaneously new according to corresponding to the specified journal file Filename, old file name and original offset, it is corresponding from journal file corresponding to the old file name and the new file name Journal file in collector journal data, including：Using the current document misregistration amount of the specified journal file and filename as the original offset and described Old file name is recorded；The current document misregistration amount of the specified journal file is reset to 0, collects the daily record number in the specified journal file According to regenerating and record the file fingerprint of the specified journal file；When judging to there are the record of new file name and old file name corresponding to the specified journal file, according to the original text Part offset, the collector journal data from journal file corresponding to the new file name.
- 5. the method as described in claim 1-4 any claims, it is characterised in that described obtain specifies journal file to be write After the event for entering daily record data, judge the current file fingerprint of the specified journal file, with the specified journal file it Before whether preceding file fingerprint is consistent, in addition to：Judge whether the specified journal file is collected daily record data for the first time；Collected if it is, the specified journal file is added in file set, the file of the specified journal file is inclined Shifting amount is arranged to 0, collects the daily record data in the specified journal file, generates and record the file of the specified journal file Fingerprint；If it is not, then the file fingerprint for judging that the specified journal file is current is performed, with the specified journal file The whether consistent step of file fingerprint before.
- 6. method as claimed in claim 5, it is characterised in that the file for judging that the specified journal file is current refers to After whether file fingerprint before line, with the specified journal file is consistent, in addition to：If consistent, according to the current document misregistration amount of the specified journal file, collected from the specified journal file Daily record data, and reset after collector journal data the document misregistration amount of the specified journal file.
- 7. a kind of device of collector journal data, it is characterised in that described device includes：Monitoring modular, for the JFS specified by the monitoring of file change notification interface；First acquisition module, the event of daily record data is written into for obtaining specified journal file；First judge module, the file fingerprint current for judging the specified journal file, with the specified journal file it Whether preceding file fingerprint is consistent, and the file fingerprint is according to the predetermined word joint number collected from the specified journal file What word determined；First collection module, if the judged result for first judge module is inconsistent, obtain and according to described New file name, old file name and original offset corresponding to journal file are specified, from daily record text corresponding to the old file name Collector journal data in journal file corresponding to part and the new file name.
- 8. device as claimed in claim 7, it is characterised in that described device also includes：Second acquisition module, the event being renamed for obtaining the specified journal file；Logging modle, for recording new file name and old file name corresponding to the specified journal file.
- 9. device as claimed in claim 8, it is characterised in that the file change notification interface includes：In linux system File system monitoring FileSystemWatcher in inotify, windows system；Correspondingly, the event that the specified journal file is written into daily record data includes：File in inotify has new write-in In_modify events, the change Changed events in FileSystemWatcher；Correspondingly, the event that the specified journal file is renamed includes：File in inotify removes in_move_from Event and file move into in_move_to events, the renaming Renamed events in FileSystemWatcher.
- 10. device as claimed in claim 8, it is characterised in that first collection module includes：Recording unit, if for inconsistent, the current document misregistration amount of the specified journal file and filename are made respectively Recorded for the original offset and the old file name；Processing unit, for the current document misregistration amount of the specified journal file to be reset into 0, collect the specified daily record text Daily record data in part, regenerate and record the file fingerprint of the specified journal file；Collector unit, for there are the record of new file name and old file name corresponding to the specified journal file when judgement When, according to the original offset, the collector journal data from journal file corresponding to the new file name.
- 11. the device as described in claim 7-10 any claims, it is characterised in that described device also includes：Second judge module, for obtaining after specified journal file is written into the event of daily record data, judge the specified day Before whether file fingerprint before the current file fingerprint of will file, with the specified journal file is consistent, the finger is judged Determine whether journal file is collected daily record data for the first time；Processing module, if the judged result for second judge module is, the specified journal file is added to Collect in file set, the document misregistration amount of the specified journal file is arranged to 0, collected in the specified journal file Daily record data, generate and record the file fingerprint of the specified journal file；Notification module, if the judged result for second judge module is to notify first judge module The file fingerprint for judging that the specified journal file is current is performed, is with the file fingerprint before the specified journal file No consistent step.
- 12. device as claimed in claim 11, it is characterised in that described device also includes：Second collection module, if the judged result for first judge module is consistent, according to the specified daily record The current document misregistration amount of file, the collector journal data from the specified journal file, and after collector journal data again The document misregistration amount of the specified journal file is set.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310676525.2A CN104714878B (en) | 2013-12-11 | 2013-12-11 | A kind of method and device of collector journal data |
| HK15107341.4A HK1206838A1 (en) | 2013-12-11 | 2015-07-31 | Method for collecting log data and device thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310676525.2A CN104714878B (en) | 2013-12-11 | 2013-12-11 | A kind of method and device of collector journal data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104714878A CN104714878A (en) | 2015-06-17 |
| CN104714878B true CN104714878B (en) | 2017-11-21 |
Family
ID=53414237
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310676525.2A Active CN104714878B (en) | 2013-12-11 | 2013-12-11 | A kind of method and device of collector journal data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN104714878B (en) |
| HK (1) | HK1206838A1 (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105955876B (en) * | 2016-05-13 | 2020-08-14 | 腾讯科技（深圳）有限公司 | Data monitoring processing method and device |
| CN106250496A (en) * | 2016-08-02 | 2016-12-21 | 北京集奥聚合科技有限公司 | A kind of method and system of the data collection in journal file |
| CN108062323B (en) * | 2016-11-08 | 2021-10-15 | 北京国双科技有限公司 | Log reading method and device |
| CN108038253B (en) * | 2018-01-23 | 2021-03-12 | 奇安信科技集团股份有限公司 | Log query processing method and device |
| CN108733546A (en) * | 2018-04-02 | 2018-11-02 | 阿里巴巴集团控股有限公司 | A kind of log collection method, device and equipment |
| CN110378154B (en) * | 2019-07-24 | 2021-04-02 | 秒针信息技术有限公司 | File set integrity checking method and device |
| CN112905106B (en) * | 2019-12-04 | 2023-04-18 | 贵州白山云科技股份有限公司 | Data processing method, device, system, equipment and storage medium |
| CN111856249B (en) * | 2020-07-24 | 2022-12-13 | 安测半导体技术(江苏)有限公司 | Chip test monitoring method, client and system |
| CN112506884A (en) * | 2020-12-10 | 2021-03-16 | 杭州安恒信息技术股份有限公司 | Log checking method, device, equipment and storage medium |
| CN114996335B (en) * | 2022-08-03 | 2022-11-01 | 海看网络科技(山东)股份有限公司 | IPTV log real-time clustering analysis method |
| CN117056288A (en) * | 2023-08-17 | 2023-11-14 | 齐鲁空天信息研究院 | Method and system for searching and downloading server file |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1159649C (en) * | 2000-12-28 | 2004-07-28 | 中兴通讯股份有限公司 | Automatic circular daily record information storing method and system |
| JP2008198039A (en) * | 2007-02-15 | 2008-08-28 | Hitachi Ltd | Log collection and management method |
| CN101561807A (en) * | 2008-04-18 | 2009-10-21 | 北京闻言科技有限公司 | Method for recording logs with file by flexibly planning catalog |
| JP2013191012A (en) * | 2012-03-14 | 2013-09-26 | Hitachi Ltd | Computer log collection system and attachment/detachment type log storage device |
| CN102880718B (en) * | 2012-10-12 | 2015-09-02 | 互动在线（北京）科技有限公司 | A kind of storage of flexible daily record and acquisition methods |
| CN102929998A (en) * | 2012-10-25 | 2013-02-13 | 浪潮(北京)电子信息产业有限公司 | File increment intercepting method and device |
-
2013
- 2013-12-11 CN CN201310676525.2A patent/CN104714878B/en active Active
-
2015
- 2015-07-31 HK HK15107341.4A patent/HK1206838A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| HK1206838A1 (en) | 2016-01-15 |
| CN104714878A (en) | 2015-06-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104714878B (en) | A kind of method and device of collector journal data | |
| CN104199750B (en) | A kind of file access pattern method and device of Linux system | |
| CN109448100B (en) | Three-dimensional model format conversion method, system, computer device and storage medium | |
| Al Kez et al. | Exploring the sustainability challenges facing digitalization and internet data centers | |
| CN103001796A (en) | Method and device for processing weblog data by server | |
| CN109784058A (en) | Version strong consistency method of calibration, client, server and storage medium | |
| CN101430691A (en) | System and method for exporting data directly from deduplication storage to non-deduplication storage | |
| US11977532B2 (en) | Log record identification using aggregated log indexes | |
| CN103793493A (en) | Method and system for processing car-mounted terminal mass data | |
| CN106844089B (en) | Method and equipment for recovering tree data storage | |
| CN105260294A (en) | Monitoring method and device for memory occupied by application program | |
| CN105573859A (en) | Data recovery method and device of database | |
| Bobade | Survey paper on big data and Hadoop | |
| CN109271361B (en) | Distributed storage method and system for massive small files | |
| CN104239438A (en) | File information storage method and file information read-write method based on separate storage | |
| CN105446825A (en) | Database test method and device | |
| CN104036034A (en) | Log analysis method and device for data warehouse | |
| US20100106681A1 (en) | Identifying Files Associated With A Workflow | |
| CN108959359A (en) | A kind of uniform resource locator semanteme De-weight method, device, equipment and medium | |
| CN103412929A (en) | Mass data storage method | |
| CN109033365B (en) | Data processing method and related equipment | |
| US20240070580A1 (en) | Systems and Methods for Decommissioning Business Intelligence Artifacts | |
| CN106354587A (en) | Mirror image server and method for exporting mirror image files of virtual machine | |
| CN106445732B (en) | A kind of online snapshot management method and system based on Version Control | |
| CN109189343B (en) | Metadata disk-dropping method, device, equipment and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1206838 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1206838 Country of ref document: HK |