CN117271445A - Log data processing method, device, server, storage medium and program product - Google Patents

Log data processing method, device, server, storage medium and program product Download PDF

Info

Publication number
CN117271445A
CN117271445A CN202311175241.5A CN202311175241A CN117271445A CN 117271445 A CN117271445 A CN 117271445A CN 202311175241 A CN202311175241 A CN 202311175241A CN 117271445 A CN117271445 A CN 117271445A
Authority
CN
China
Prior art keywords
log
target
log data
serial number
service
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311175241.5A
Other languages
Chinese (zh)
Inventor
闫迎兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bank of China Ltd
Original Assignee
Bank of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bank of China Ltd filed Critical Bank of China Ltd
Priority to CN202311175241.5A priority Critical patent/CN117271445A/en
Publication of CN117271445A publication Critical patent/CN117271445A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/14—Details of searching files based on file metadata
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G06F11/3476—Data logging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815—Journaling file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/02—Banking, e.g. interest calculation or account maintenance

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Data Mining & Analysis (AREA)
  • Finance (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Library & Information Science (AREA)
  • Computer Hardware Design (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application relates to a log data processing method, a log data processing device, a server, a storage medium and a program product, which can be used in the technical field of big data. The method comprises the following steps: determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number; determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers; and carrying out log analysis processing according to the target log file to obtain a target analysis result. According to the embodiment of the application, the time for log query analysis can be saved, so that the log query analysis efficiency can be improved.

Description

Log data processing method, device, server, storage medium and program product
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a log data processing method, device, server, storage medium, and program product.
Background
With the rapid development of internet technology and the acceleration of digital transformation of banks, the amount of log data of business systems is rapidly increasing. The log data generated by the business system can be used as key data for evaluating the performance of the system, monitoring the safety and/or solving the problems of the system, so how to acquire the needed log data from massive log data is of great importance to the management process of the business system.
In the related art, log data in a service system are usually mixed and recorded together according to a time sequence, and when in query, required log data are usually searched from massive log data according to query time. But the log query analysis in the related art is inefficient.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a log data processing method, apparatus, server, storage medium, and program product that can improve the efficiency of log query analysis.
In a first aspect, the present application provides a log data processing method. The method comprises the following steps:
determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number;
Determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers;
and carrying out log analysis processing according to the target log file to obtain a target analysis result.
In one embodiment, log analysis processing is performed according to a target log file to obtain a target analysis result, including:
determining each target log data from the target log file according to the auxiliary keywords to be queried;
and carrying out log analysis processing according to each target log data to obtain a target analysis result.
In one embodiment, the target log file further includes an interface calling sequence, and the log analysis processing is performed according to the target log file to obtain a target analysis result, including:
and carrying out log analysis processing according to the interface calling sequence in the target log file to obtain a target analysis result.
In one embodiment, the method further comprises:
acquiring a plurality of pieces of log data to be classified;
and storing the plurality of pieces of log data in a slicing way according to the service processing serial numbers to obtain a plurality of log files.
In one embodiment, the storing the plurality of pieces of log data in slices according to the serial numbers of the service processing to obtain a plurality of log files includes:
dividing a plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to each log data in the log data set are the same;
for each log data set, storing each log data in the log data set into a corresponding log file; wherein the log file is associated with a business process serial number.
In one embodiment, if part of the log data in the log data set includes interface call information, the method further includes:
according to the interface calling information in the log data set, determining an interface calling sequence corresponding to the log data set;
and storing the interface calling sequence to a log file corresponding to the log data set.
In a second aspect, the present application further provides a log data processing apparatus. The device comprises:
the first determining module is used for determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number;
The second determining module is used for determining a target log file corresponding to the target business processing serial number from the plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers;
and the analysis module is used for carrying out log analysis processing according to the target log file to obtain a target analysis result.
In a third aspect, the present application also provides a server. The server comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of the method of the first aspect described above.
In a fourth aspect, the present application also provides a computer-readable storage medium. A computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the method of the first aspect described above.
In a fifth aspect, the present application also provides a computer program product. Computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of the first aspect described above.
According to the log data processing method, the log data processing device, the server, the storage medium and the program product, the target service processing serial number corresponding to the service identification is determined according to the service identification to be queried and the corresponding relation between the preset service identification and the service processing serial number; determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers; and carrying out log analysis processing according to the target log file to obtain a target analysis result. In the embodiment of the present application, the target log file corresponding to the target service processing serial number is determined from the plurality of log files according to the target service processing serial number corresponding to the service identifier to be queried, because the target log file in the embodiment of the present application includes each log data corresponding to the target service processing serial number, compared with the manner of querying log data from massive log data in the related art, the number of log data in the target log file in the embodiment of the present application is relatively small, so that the time of log query analysis can be saved based on each log data in the target log file, thereby improving the log query analysis efficiency.
Drawings
FIG. 1 is a schematic view of an application environment of a log data processing method according to an embodiment of the present application;
FIG. 2 is a flowchart of a log data processing method according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating a log data processing method according to another embodiment of the present disclosure;
FIG. 4 is a flowchart of a log data processing method according to another embodiment of the present application;
FIG. 5 is a flowchart of a log data processing method according to another embodiment of the present application;
FIG. 6 is a flowchart of a log data processing method according to another embodiment of the present application;
FIG. 7 is a schematic diagram of a log data processing apparatus according to an embodiment of the present application;
fig. 8 is an internal structural diagram of a server in one embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The log data processing method, the device, the server, the storage medium and the program product provided by the embodiment of the application can be applied to an application scene of log query analysis of a service system; of course, the present invention can also be applied to other scenarios, which are not limited in the embodiments of the present application.
For convenience of explanation, the following embodiments take a log query analysis scenario in which the log data processing method of the embodiments of the present application is applied to a banking system as an example. It should be understood that when the log data processing method of the embodiment of the present application is applied to other scenarios, the implementation principle and technical effects are similar.
Fig. 1 is an application environment schematic diagram of a log data processing method according to an embodiment of the present application, and as shown in fig. 1, the application environment schematic diagram according to an embodiment of the present application may include, but is not limited to: a terminal 10 and a server 11; wherein the terminal 10 can communicate with the server 11 via a network. For example, the terminal 10 may transmit a service request or a trigger request, etc., to the server 11. The data storage system may store log data that the server 11 needs to process. The data storage system may be integrated on the server 11 or may be placed on a cloud or other network server. The server 11 in the embodiment of the present application may perform log query analysis processing by using the log data processing method in the embodiment of the present application.
By way of example, the terminals 10 may include, but are not limited to, customer-corresponding terminals or terminals that may correspond to the operation and maintenance personnel of the business system. The terminal 10 may include, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers, among others. The server 11 may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.
In the related art, log data in a service system are usually recorded together in a mixed manner according to a time sequence, and when a log is queried or a bit is determined, the required log data are often required to be searched from massive log data according to the query time. However, since there may be a difference between the server time and the real environment time, the time stamp in the log data may not accurately reflect the time information corresponding to the service processing, and thus, it may be difficult to retrieve the log based on the query time, resulting in low efficiency of the log query analysis.
According to the log data processing method, device, server, storage medium and program product, the target log files corresponding to the target service processing serial numbers are determined from the plurality of log files according to the target service processing serial numbers corresponding to the service identification to be queried, and because the target log files in the embodiment of the application comprise the log data corresponding to the target service processing serial numbers, compared with the mode of querying log data from massive log data in the related art, the number of the log data in the target log files in the embodiment of the application is relatively small, so that the time for log query analysis can be saved based on the log data in the target log files, and the log query analysis efficiency can be improved.
In one embodiment, fig. 2 is a flowchart of a log data processing method according to one embodiment of the present application, where the embodiment of the present application is described by taking the application of the method to the server in fig. 1 as an example. As shown in fig. 2, the method of the embodiment of the present application may include the following steps:
step S201, determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number.
Illustratively, the service identifier referred to in the embodiments of the present application may be used to uniquely indicate the service to be queried. For example, the service identification may include, but is not limited to, a service ticket number, a service flow number, a customer identification, etc., where the customer identification may include, but is not limited to, a customer identification number.
For example, the correspondence between the preset service identifier and the service processing serial number in the embodiment of the present application may be used to indicate the correspondence between the multiple sets of service identifiers and the corresponding service processing serial numbers. For example, the corresponding relationship between the preset service identifier and the service processing serial number may be used to indicate the corresponding relationship between the service identifier a and the service processing serial number 1, the corresponding relationship between the service identifier B and the service processing serial number 2, and the corresponding relationship between the service identifier C and the service processing serial number C.
Illustratively, any service processing serial number in the embodiments of the present application may be used to indicate service processing information of a corresponding service. The service processing serial numbers corresponding to the respective service processes of any service are the same.
In one possible implementation manner, the server may determine, at intervals of a preset duration, a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and a corresponding relationship between the preset service identifier and the service processing serial number; the service identifier to be queried may be a service identifier in a preset service identifier set to be queried.
In another possible implementation manner, the server may determine, when receiving the trigger request, a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and a corresponding relationship between a preset service identifier and the service processing serial number; wherein the trigger request may include, but is not limited to, a service identifier. For example, the trigger request may be a log query request or an error indication message, etc.
Of course, the server may also obtain the service identifier to be queried in other manners.
In this step, the server may query the corresponding relationship between the preset service identifier and the service processing serial number according to the service identifier to be queried, and take the service processing serial number corresponding to the service identifier to be queried as the target service processing serial number.
For example, assuming that the correspondence between the preset service identifier and the service processing serial number may be used to indicate the correspondence between the service identifier a and the service processing serial number 1 and the correspondence between the service identifier B and the service processing serial number 2, the service identifier to be queried is the service identifier a, the server may query the correspondence between the preset service identifier and the service processing serial number according to the service identifier a to be queried, and take the service processing serial number 1 corresponding to the service identifier a to be queried as the target service processing serial number.
Step S202, determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number.
Different log files in the embodiment of the application are used for storing log data corresponding to different service processing serial numbers, wherein the service processing serial numbers of the log data in any log file are the same.
By way of example, log data in embodiments of the present application may include, but is not limited to, business process data, server operating state data, error exception data, terminal behavior demand data.
In this step, the server may determine, from the plurality of log files, a target log file corresponding to the target service processing serial number according to the target service processing serial number; the target log file stores log data of each service process of the target service corresponding to the target service processing serial number.
In a possible implementation manner, any log file in the embodiment of the present application is associated with a corresponding service processing serial number, and the server may match the target service processing serial number with the service processing serial number associated with each log file, and use the log file associated with the service processing serial number matched with the target service processing serial number as the target log file.
In another possible implementation manner, the server may query log index information according to the target service processing serial number, determine a log file index corresponding to the target service processing serial number, and determine a target log file according to the log file index; the log index information may include a plurality of sets of records of service processing serial numbers and corresponding log file indexes, and any log file index may include, but is not limited to, name information or storage location information corresponding to a log file.
Of course, the server may determine, from the plurality of log files, the target log file corresponding to the target service processing serial number according to the target service processing serial number in other manners.
And step S203, performing log analysis processing according to the target log file to obtain a target analysis result.
In this step, the server may perform log analysis processing according to the target log file to obtain a target analysis result. Illustratively, the log analysis results in embodiments of the present application may include, but are not limited to, problem location results, or test analysis results.
Because the target log file in the embodiment of the application includes the log data of each service process of the target service corresponding to the target service processing serial number, compared with the mode of inquiring the log data from massive log data in the related technology, the number of the log data in the target log file in the embodiment of the application is relatively small, so that the server can quickly perform log analysis processing based on each log data in the target log file.
In one possible implementation, determining each target log data from the target log file according to the auxiliary keyword to be queried; and carrying out log analysis processing according to each target log data to obtain a target analysis result.
The auxiliary keywords in the embodiment of the application may be carried in the trigger request, or may be obtained by querying a correspondence between a preset service identifier and the keywords according to the service identifier to be queried by the server; of course, the affiliated keywords may also be obtained by other means for the server.
It should be noted that, in the embodiments of the present application, the auxiliary keywords are collectively referred to, and may include one keyword, or a plurality of keywords.
In this implementation manner, the server may use, according to the auxiliary keyword to be queried, each log data corresponding to the auxiliary keyword in the target log file as target log data, and perform log analysis processing according to each target log data, to obtain a target analysis result.
For example, the server may use a preset search method to screen out each target log data corresponding to the auxiliary keyword from the target log file.
Therefore, in the implementation manner, each log data in the target log file is further screened according to the auxiliary keywords, so that log analysis processing can be more rapidly performed based on each screened target log data, and the log query analysis efficiency is further improved.
In another possible implementation manner, the target log file further includes an interface calling sequence, and log analysis processing is performed according to the interface calling sequence in the target log file, so as to obtain a target analysis result.
In the service processing process for any service, the server in the embodiment of the application generally communicates with other servers or devices by calling corresponding interfaces.
The interface calling sequence in the embodiment of the application can be used for indicating the sequence of the interface calling of the server in the service processing process.
In the implementation manner, the server can acquire the sequence of the interface call and related log data of the server in the service processing process aiming at the target service according to the sequence of the interface call in the target log file, and perform log analysis processing based on the sequence of the interface call and the related log data. Because the interface calling sequence is stored in the log file in the embodiment of the application, the interface calling information can be acquired based on the log file, so that a user can conveniently check and trace back the calling interface without special positioning of experienced personnel or personnel familiar with business scenes, and the embodiment of the application is favorable for further saving the time of log query analysis.
It should be noted that the two implementations may be combined with each other, for example, target log data meeting the conditions may be first screened according to the auxiliary keyword query, and then interface calling sequence information may be combined when log analysis is performed on the target log data.
In the log data processing method, the target service processing serial number corresponding to the service identifier is determined according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number. Further, determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number, and performing log analysis processing according to the target log file to obtain a target analysis result; and the different log files are used for storing log data corresponding to different service processing serial numbers. It can be seen that, in the embodiment of the present application, by determining, from a plurality of log files, the target log file corresponding to the target service processing serial number according to the target service processing serial number corresponding to the service identifier to be queried, because the target log file in the embodiment of the present application includes each log data corresponding to the target service processing serial number, compared with the manner of querying log data from massive log data in the related art, the number of log data in the target log file in the embodiment of the present application is relatively small, so that time for log query analysis can be saved based on each log data in the target log file, thereby improving log query analysis efficiency.
In one embodiment, fig. 3 is a schematic flow chart of a log data processing method according to another embodiment of the present application, and on the basis of the foregoing embodiment, in the embodiment of the present application, description is made on a manner of acquiring a plurality of log files related to the foregoing embodiment. As shown in fig. 3, the method of the embodiment of the present application may further include the following steps:
step S301, acquiring a plurality of pieces of log data to be classified.
In this step, the server may acquire the plurality of pieces of log data to be classified that have been generated, or may generate the plurality of pieces of log data to be classified in real time.
In one possible implementation manner, the server may generate corresponding log data in real time during the service processing process for each service, so as to record the running condition of the server during the service processing process, so as to help the developer or the operation and maintenance personnel to quickly locate the problem occurring during the service processing process based on the log data.
In another possible implementation, the server may obtain, from a local or other device, a plurality of pieces of log data that have been generated to be categorized.
In this implementation manner, the server may locally obtain, through the query interface, a plurality of pieces of log data that meet the preset query condition, or may obtain, through a communication interface with other devices, a plurality of pieces of log data that meet the preset query condition from other devices.
Therefore, in the implementation manner, the server can acquire the plurality of logs meeting the conditions from the corresponding positions through the preset query conditions or the preset acquisition conditions, so that the acquisition efficiency of the log data can be improved.
Of course, the server may also obtain the pieces of log data to be classified in other manners.
And step S302, storing the plurality of pieces of log data in a slicing way according to the serial numbers of the business processing to obtain a plurality of log files.
In this step, the server may store the plurality of pieces of log data obtained in the step S301 in a slicing manner according to the service processing serial number, so as to obtain a plurality of log files; wherein, the business processing serial numbers of the log data in any log file are the same.
For example, assuming that the log data D1 corresponds to the service processing serial number 1, the log data D2 corresponds to the service processing serial number 2, the log data D3 corresponds to the service processing serial number 1, the log data D4 corresponds to the service processing serial number 3, the server may store the log data D1 and the log data D3 to the log file L1, the log data D2 to the log file L2, and the log data D4 to the log file L3 according to the service processing serial number.
Optionally, dividing the plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to each log data in the log data set are the same; for each log data set, storing each log data in the log data set into a corresponding log file; wherein the log file is associated with a business process serial number. In this embodiment of the present invention, the server may divide each log data with the same service processing serial number into the same log data set to obtain a plurality of log data sets, where service processing serial numbers corresponding to different log data sets are different.
For example, assuming that the log data D1 corresponds to the service processing serial number 1, the log data D2 corresponds to the service processing serial number 2, the log data D3 corresponds to the service processing serial number 1, the log data D4 corresponds to the service processing serial number 3, the server may divide the log data D1 and the log data D3 into the log data set S1, the log data D2 into the log data set S2, and the log data D4 into the log data set S3 according to the service processing serial number.
Further, for each log data set, the server may store each log data in the log data set to a corresponding log file; the log file is associated with the service processing serial number, so that the corresponding log file can be queried through the service processing serial number later.
For example, the server may store each log data in the log data set S1 to the log file L1, each log data in the log data set S2 to the log file L2, and the log data set S3 to the log file L3.
Therefore, in the embodiment of the application, the plurality of pieces of log data are divided into the plurality of log data sets according to the service processing serial numbers, and each piece of log data in each log data set is respectively stored in the corresponding log file, so that the log data can be stored in a slicing manner according to the service processing serial numbers, the corresponding log files can be quickly positioned based on the service processing serial numbers, and log analysis processing can be quickly performed based on each piece of log data in the log files.
Based on the above embodiment, if part of the log data in any log data set includes interface call information, the log data processing method in the embodiment of the present application may further include:
according to the interface calling information in the log data set, determining an interface calling sequence corresponding to the log data set; and storing the interface calling sequence to a log file corresponding to the log data set.
Illustratively, the interface call information in the embodiment of the application may be used to indicate interface call information of the server in a service processing process, where the interface call information may include, but is not limited to, an identifier of an interface and/or a call time of the interface.
According to the embodiment of the application, the server can determine the interface calling sequence corresponding to the log data set according to the interface calling information in the log data set, and store the interface calling sequence into the log file corresponding to the log data set, so that the calling sequence of each interface and related log data can be clearly known based on the interface calling sequence, and the log query analysis efficiency can be improved.
In summary, in the embodiment of the present application, by acquiring a plurality of pieces of log data to be classified and storing the plurality of pieces of log data in a slicing manner according to a service processing serial number, a corresponding log file can be quickly located and traced based on the service processing serial number, and log analysis processing can be quickly performed based on each piece of log data in the log file.
In one embodiment, fig. 4 is a schematic flow chart of a log data processing method according to another embodiment of the present application, and as shown in fig. 4, the overall flow of the log data processing method according to the embodiment of the present application is described based on the embodiment. As shown in fig. 4, the method of the embodiment of the present application may further include the following steps:
s401, the server can acquire a plurality of pieces of log data to be classified.
S402, the server can store a plurality of pieces of log data in a slicing mode according to the service processing serial numbers to obtain a plurality of log files.
S403, the server can determine the target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number.
S404, the server can determine a target log file corresponding to the target service processing serial number from a plurality of log files according to the target service processing serial number.
S405, the server can perform log analysis processing according to the target log file to obtain a target analysis result.
It should be noted that, in the embodiments of the present application, the realizable manner of each step may refer to the related content in the foregoing embodiments, which is not described herein again.
In summary, in the embodiment of the present application, by storing log data in slices according to the service processing serial numbers, positioning a target log file according to the target service processing serial numbers corresponding to the service identifier to be queried, and performing log analysis processing according to the target log file, the time of log query analysis can be greatly saved, so that the log query analysis efficiency can be improved, and the problem positioning efficiency and the problem solving efficiency are improved.
In an embodiment, fig. 5 is a schematic flow chart of a log data processing method according to another embodiment of the present application, and as shown in fig. 5, in the embodiment of the present application, a server includes a log data obtaining module and a slice storage module, and related contents of a log data storage process in the log data processing method are described. As shown in fig. 5, the method of the present embodiment may include the steps of:
1) The log data acquisition module acquires a plurality of pieces of log data to be classified.
The log data in the embodiment of the application may include, but is not limited to, log data obtained by a server in a process of performing service processing according to a service request sent by a terminal through a communication interface.
2) And the slicing storage module performs slicing storage on the plurality of pieces of log data according to the service processing serial numbers to obtain a plurality of log files.
For example, different log files may be stored on different memory modules (or memory partitions). For convenience of illustration, three storage modules corresponding to the three log files are shown in fig. 5 as an example.
It should be noted that, the realizable manner and the technical effect of each step in the embodiments of the present application may refer to the relevant content in the above embodiments, which is not repeated herein.
In one embodiment, fig. 6 is a schematic flow chart of a log data processing method according to another embodiment of the present application, and as shown in fig. 6, in the embodiment of the present application, a server includes a serial number acquisition module and a query analysis module, and related content of a log data query analysis process in the log data processing method is described. As shown in fig. 6, the method of the present embodiment may include the steps of:
step S601, a serial number acquisition module determines a target service processing serial number corresponding to a service identifier according to the service identifier to be queried and the corresponding relation between a preset service identifier and the service processing serial number.
Step S602, a query analysis module determines a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number, and performs log analysis processing according to the target log file to obtain a target analysis result.
It should be noted that, the realizable manner and the technical effect of each step in the embodiments of the present application may refer to the relevant content in the above embodiments, which is not repeated herein.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a log data processing device for realizing the above related log data processing method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of one or more log data processing devices provided below may refer to the limitation of the log data processing method in the above description, and will not be repeated here.
In one embodiment, fig. 7 is a schematic structural diagram of a log data processing device according to one embodiment of the present application, where the log data processing device provided in the embodiment of the present application may be applied to a server. As shown in fig. 7, the log data processing apparatus according to the embodiment of the present application may include: a first determination module 701, a second determination module 702, and an analysis module 703.
The first determining module 701 is configured to determine, according to a service identifier to be queried and a corresponding relationship between a preset service identifier and a service processing serial number, a target service processing serial number corresponding to the service identifier;
a second determining module 702, configured to determine, from a plurality of log files, a target log file corresponding to the target service processing serial number according to the target service processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers;
And the analysis module 703 is used for performing log analysis processing according to the target log file to obtain a target analysis result.
In one embodiment, the analysis module 703 is specifically configured to:
determining each target log data from the target log file according to the auxiliary keywords to be queried;
and carrying out log analysis processing according to each target log data to obtain a target analysis result.
In one embodiment, the target log file further includes an interface calling sequence, and the analysis module 703 is specifically configured to:
and carrying out log analysis processing according to the interface calling sequence in the target log file to obtain a target analysis result.
In one embodiment, the log data processing apparatus further comprises: the device comprises an acquisition module and a storage module.
The acquisition module is used for acquiring a plurality of pieces of log data to be classified;
and the storage module is used for storing the plurality of pieces of log data in a slicing way according to the service processing serial numbers to obtain a plurality of log files.
In one embodiment, the storage module may include: a dividing unit and a storage unit.
The dividing unit is used for dividing the plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to each log data in the log data set are the same;
The storage unit is used for storing each log data in each log data set into a corresponding log file; wherein the log file is associated with a business process serial number.
In one embodiment, if the portion of the log data in the log data set includes interface call information, the storage unit is further configured to:
according to the interface calling information in the log data set, determining an interface calling sequence corresponding to the log data set;
and storing the interface calling sequence to a log file corresponding to the log data set.
The log data processing device provided in the embodiment of the present application may be used to execute the technical scheme in the embodiment of the log data processing method, and its implementation principle and technical effect are similar, and are not repeated here.
The various modules in the log data processing apparatus described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or independent of a processor in a server, or may be stored in software in a memory in the server, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, fig. 8 is an internal structure diagram of a server in one embodiment of the present application, and as shown in fig. 8, the server in the embodiment of the present application may include a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the server is configured to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the server is used for exchanging information between the processor and the external device. The communication interface of the server is used for communicating with external equipment through network connection. The computer program, when executed by a processor, implements the log data processing method in the above-described embodiments of the present application.
It will be appreciated by those skilled in the art that the structure shown in fig. 8 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the server to which the present application is applied, and that a particular server may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a server is provided that includes a memory and a processor, the memory having a computer program stored therein, the processor when executing the computer program performing the steps of:
determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number;
determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers;
and carrying out log analysis processing according to the target log file to obtain a target analysis result.
In one embodiment, the processor when executing the computer program further performs the steps of:
Determining each target log data from the target log file according to the auxiliary keywords to be queried; and carrying out log analysis processing according to each target log data to obtain a target analysis result.
In one embodiment, the target log file further includes an interface calling sequence, and the processor when executing the computer program further implements the following steps:
and carrying out log analysis processing according to the interface calling sequence in the target log file to obtain a target analysis result.
In one embodiment, the processor when executing the computer program further performs the steps of:
acquiring a plurality of pieces of log data to be classified; and storing the plurality of pieces of log data in a slicing way according to the service processing serial numbers to obtain a plurality of log files.
In one embodiment, the processor when executing the computer program further performs the steps of:
dividing a plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to each log data in the log data set are the same; for each log data set, storing each log data in the log data set into a corresponding log file; wherein the log file is associated with a business process serial number.
In one embodiment, if a portion of the log data in the log data set includes interface call information, the processor when executing the computer program further performs the steps of:
according to the interface calling information in the log data set, determining an interface calling sequence corresponding to the log data set; and storing the interface calling sequence to a log file corresponding to the log data set.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number;
determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers;
and carrying out log analysis processing according to the target log file to obtain a target analysis result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Determining each target log data from the target log file according to the auxiliary keywords to be queried; and carrying out log analysis processing according to each target log data to obtain a target analysis result.
In one embodiment, the target log file further includes an interface calling sequence, and the computer program when executed by the processor further implements the steps of:
and carrying out log analysis processing according to the interface calling sequence in the target log file to obtain a target analysis result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a plurality of pieces of log data to be classified; and storing the plurality of pieces of log data in a slicing way according to the service processing serial numbers to obtain a plurality of log files.
In one embodiment, the computer program when executed by the processor further performs the steps of:
dividing a plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to each log data in the log data set are the same; for each log data set, storing each log data in the log data set into a corresponding log file; wherein the log file is associated with a business process serial number.
In one embodiment, if a portion of the log data in the log data set includes interface call information, the computer program when executed by the processor further performs the steps of:
according to the interface calling information in the log data set, determining an interface calling sequence corresponding to the log data set; and storing the interface calling sequence to a log file corresponding to the log data set.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number;
determining a target log file corresponding to the target business processing serial number from a plurality of log files according to the target business processing serial number; wherein, different log files are used for storing log data corresponding to different business processing serial numbers;
and carrying out log analysis processing according to the target log file to obtain a target analysis result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Determining each target log data from the target log file according to the auxiliary keywords to be queried; and carrying out log analysis processing according to each target log data to obtain a target analysis result.
In one embodiment, the target log file further includes an interface calling sequence, and the computer program when executed by the processor further implements the steps of:
and carrying out log analysis processing according to the interface calling sequence in the target log file to obtain a target analysis result.
In one embodiment, the computer program when executed by the processor further performs the steps of:
acquiring a plurality of pieces of log data to be classified; and storing the plurality of pieces of log data in a slicing way according to the service processing serial numbers to obtain a plurality of log files.
In one embodiment, the computer program when executed by the processor further performs the steps of:
dividing a plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to each log data in the log data set are the same; for each log data set, storing each log data in the log data set into a corresponding log file; wherein the log file is associated with a business process serial number.
In one embodiment, if a portion of the log data in the log data set includes interface call information, the computer program when executed by the processor further performs the steps of:
according to the interface calling information in the log data set, determining an interface calling sequence corresponding to the log data set; and storing the interface calling sequence to a log file corresponding to the log data set.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims (10)

1. A method of log data processing, the method comprising:
determining a target service processing serial number corresponding to a service identifier according to the service identifier to be queried and the corresponding relation between a preset service identifier and the service processing serial number;
determining a target log file corresponding to the target service processing serial number from a plurality of log files according to the target service processing serial number; the different log files are used for storing log data corresponding to different service processing serial numbers;
And carrying out log analysis processing according to the target log file to obtain a target analysis result.
2. The method according to claim 1, wherein the performing log analysis according to the target log file to obtain a target analysis result includes:
determining each target log data from the target log file according to the auxiliary keywords to be queried;
and carrying out log analysis processing according to the target log data to obtain the target analysis result.
3. The method of claim 1, wherein the target log file further includes an interface calling sequence, and the performing log analysis according to the target log file to obtain the target analysis result includes:
and carrying out log analysis processing according to the interface calling sequence in the target log file to obtain the target analysis result.
4. A method according to any one of claims 1-3, characterized in that the method further comprises:
acquiring a plurality of pieces of log data to be classified;
and storing the plurality of pieces of log data in a slicing way according to the service processing serial number to obtain a plurality of log files.
5. The method of claim 4, wherein the storing the plurality of pieces of log data in slices according to the service processing serial number to obtain a plurality of log files includes:
Dividing the plurality of pieces of log data into a plurality of log data sets according to the service processing serial numbers; wherein, business processing serial numbers corresponding to the log data in the log data set are the same;
for each log data set, storing each log data in the log data set into a corresponding log file; wherein the log file is associated with the business process serial number.
6. The method of claim 5, wherein if a portion of the log data in the set of log data includes interface call information, the method further comprises:
determining an interface calling sequence corresponding to the log data set according to the interface calling information in the log data set;
and storing the interface calling sequence to a log file corresponding to the log data set.
7. A log data processing apparatus, the apparatus comprising:
the first determining module is used for determining a target service processing serial number corresponding to the service identifier according to the service identifier to be queried and the corresponding relation between the preset service identifier and the service processing serial number;
the second determining module is used for determining a target log file corresponding to the target service processing serial number from a plurality of log files according to the target service processing serial number; the different log files are used for storing log data corresponding to different service processing serial numbers;
And the analysis module is used for carrying out log analysis processing according to the target log file to obtain a target analysis result.
8. A server comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
CN202311175241.5A 2023-09-12 2023-09-12 Log data processing method, device, server, storage medium and program product Pending CN117271445A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311175241.5A CN117271445A (en) 2023-09-12 2023-09-12 Log data processing method, device, server, storage medium and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311175241.5A CN117271445A (en) 2023-09-12 2023-09-12 Log data processing method, device, server, storage medium and program product

Publications (1)

Publication Number Publication Date
CN117271445A true CN117271445A (en) 2023-12-22

Family

ID=89213507

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311175241.5A Pending CN117271445A (en) 2023-09-12 2023-09-12 Log data processing method, device, server, storage medium and program product

Country Status (1)

Country Link
CN (1) CN117271445A (en)

Similar Documents

Publication Publication Date Title
CN111352902A (en) Log processing method and device, terminal equipment and storage medium
CN110162512B (en) Log retrieval method, device and storage medium
CN112395157B (en) Audit log acquisition method and device, computer equipment and storage medium
CN111475105A (en) Monitoring data storage method, device, server and storage medium
CN108228611B (en) Document information copying method and device
CN116561607A (en) Method and device for detecting abnormality of resource interaction data and computer equipment
CN112527813A (en) Data processing method and device of business system, electronic equipment and storage medium
CN115481026A (en) Test case generation method and device, computer equipment and storage medium
CN117271445A (en) Log data processing method, device, server, storage medium and program product
CN113778996A (en) Large data stream data processing method and device, electronic equipment and storage medium
US9483560B2 (en) Data analysis control
CN116628042A (en) Data processing method, device, equipment and medium
CN112364007B (en) Mass data exchange method, device, equipment and storage medium based on database
CN106528577B (en) Method and device for setting file to be cleaned
CN112860755B (en) Service identifier generation method, device, computer equipment and medium
CN112711608B (en) Data display method, device, computer readable storage medium and computer equipment
CN116225830A (en) Service operation log reporting method and device, computer equipment and storage medium
CN117785674A (en) Interface use case generation method, device, computer equipment and storage medium
CN117389960A (en) File parsing method, apparatus, device, storage medium and program product
CN117632009A (en) Data storage method and device based on business application and computer equipment
CN117708117A (en) Batch data processing method, device, electronic device and storage medium
CN117135077A (en) Virtual address access fault processing method, device and computer equipment
CN117635105A (en) Method, device, computer equipment and storage medium for determining rush repair scheme
CN117312283A (en) Database and table data verification method and device, computer equipment and storage medium
CN116521546A (en) Interface performance adjusting method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination