CN115167847A - Application log acquisition method, device, equipment and storage medium - Google Patents

Abstract

The embodiment of the invention discloses an application log collection method, an application log collection device, application log collection equipment and a storage medium, wherein a log collection proxy file associated with a target application is loaded in a loading mode in response to the start of the target application, and an original log collection statement in an original byte code file of the target application is regulated through a byte code regulation statement in the log collection proxy file to obtain a target log collection statement capable of collecting target log data in the running process of the target application.

Description

Application log acquisition method, device, equipment and storage medium

Technical Field

The invention relates to the technical field of computers, in particular to an application log acquisition method, device, equipment and storage medium.

Background

Currently, the development of an application project is usually performed by cooperation of multiple persons. In the application development stage, log codes inserted into application source codes by different developers are different, which may cause that logs output by the application after the development is completed do not meet business requirements, for example, a large amount of useless logs are output, or business requirement logs cannot be output, and the like, thereby wasting log storage space and reducing efficiency of checking and positioning application problems.

In order to solve the problem that the operation log output by the application does not meet the service requirement, the prior art generally adopts a hard coding mode, and depends on or inherits the log class provided by the application to modify the source code of the application; or, modifying the source code of the application by adopting a mode of facing to the section programming, so that the application can output the running log meeting the service requirement.

In the process of implementing the invention, the inventor finds that at least the following technical problems exist in the prior art: because the hard coding mode needs to rely on or inherit the log class provided by the application, the system intrusiveness is strong, and the implementation complexity is high, and the hard coding mode needs to modify the source code of the application, so that the source code of the application cannot be multiplexed. The mode of the section-oriented programming is run-time (runtime), the performance loss is large, and the source code of the application also needs to be modified.

Disclosure of Invention

The embodiment of the invention provides an application log acquisition method, device, equipment and storage medium.

According to an aspect of an embodiment of the present invention, there is provided an application log collecting method, including:

responding to the starting of a target application, and loading a log collection proxy file associated with the target application, wherein the log collection proxy file comprises a byte code adjusting statement;

based on the bytecode adjustment statement, accessing an original bytecode file of the target application, and adjusting an original log acquisition statement in the original bytecode file to obtain a target bytecode file;

and acquiring target log data corresponding to the target log acquisition statement in the running process of the target application based on the target log acquisition statement in the target byte code file.

According to another aspect of the embodiments of the present invention, there is provided an application log collecting apparatus, including:

the system comprises a proxy file loading module, a log collection proxy file loading module and a data processing module, wherein the proxy file loading module is used for responding to the starting of a target application and loading the log collection proxy file associated with the target application, and the log collection proxy file comprises a byte code adjusting statement;

the bytecode adjusting module is used for accessing the original bytecode file of the target application based on the bytecode adjusting statement and adjusting the original log acquisition statement in the original bytecode file to obtain a target bytecode file;

and the log acquisition module is used for acquiring target log data corresponding to the target log acquisition statement in the running process of the target application based on the target log acquisition statement in the target byte code file.

According to another aspect of the embodiments of the present invention, there is provided an electronic apparatus, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the application log collection method according to any embodiment of the invention.

According to another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium storing computer instructions for enabling a processor to implement the application log collection method according to any one of the embodiments of the present invention when the computer instructions are executed.

One embodiment of the above invention has the following advantages or benefits:

the method comprises the steps of responding to the starting of a target application, loading a log collection proxy file associated with the target application together, adjusting an original log collection statement in an original bytecode file of the target application through a bytecode adjustment statement in the log collection proxy file to obtain a target log collection statement capable of collecting target log data in the running process of the target application, and obtaining a log meeting business requirements in the running process of the application through adjusting the bytecode of the application.

It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart of an application log collection method according to an embodiment of the present invention;

fig. 2 is a schematic flowchart of another application log collection method according to an embodiment of the present invention;

fig. 3A is a schematic flowchart of another application log collection method according to an embodiment of the present invention;

fig. 3B is a schematic diagram of a log collection process according to an embodiment of the present invention;

fig. 3C is a schematic diagram of an application log processing process according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of an application log collecting apparatus according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 is a schematic flow diagram of an application log collection method according to an embodiment of the present invention, where this embodiment is applicable to a developed application, and in order to obtain a log that meets a service requirement during an application running process, a pre-configured log collection agent file is loaded together when the application is started, so as to adjust a bytecode file of the application, so as to obtain a log that meets the service requirement during the application running process. As shown in fig. 1, the method includes:

and S110, responding to the starting of the target application, and loading a log collection proxy file associated with the target application, wherein the log collection proxy file comprises a byte code adjusting statement.

The target application may be a pre-developed application. For example, the target application may be a monolithic application or a distributed application. The log collection agent file associated with the target application may be a file generated in advance and attached to the target application, and when the target application is started, the log collection agent file is loaded together.

For example, a source code file of the target application and a log collection agent file associated with the target application may be configured in the same server in advance, a loading command for loading the log collection agent file when the target application is started may be set, and the log collection agent file may be loaded when the target application is detected to be started according to the loading command.

In this embodiment, the log collection agent file may be used to adjust the raw bytecode file of the target application. Illustratively, the target collection agent file may be a file in the form of a jar package. Specifically, the log collection agent file includes a bytecode adjustment statement. After the target collection agent file is loaded, the original bytecode file of the target application can be adjusted through the bytecode adjustment statement in the target collection agent file. The bytecode adjustment statement can be used for deleting, modifying or inserting information in an original bytecode file of the target application.

It should be noted that, in this embodiment, in response to the start of the target application, the purpose of loading the log collection agent file for adjusting the raw bytecode file of the target application is to: the original byte code file of the target application is adjusted through the byte code adjusting statement in the log collection agent file, and then the log code collection logic in the target application is adjusted, so that the target application can output logs meeting the service requirements, and the source code of the target application is prevented from being adjusted.

In this embodiment, a corresponding bytecode adjustment statement may be configured in advance according to a source code of the target application, and then a log collection proxy file is generated according to the bytecode adjustment statement.

In a specific implementation manner, the method provided in this embodiment further includes: determining a key statement for connecting the middleware corresponding to the target application in the source code of the target application; a bytecode adjustment statement is determined based on the key statement and the source code.

The middleware corresponding to the target application may be a relational Database, a distributed Database (e.g., hadoop Database), a key-value pair Database, an index Database (e.g., elasticsearch Database), or the like. The key statement may be a statement in the source code for connecting to the middleware, such as a key path of the middleware. Specifically, each key statement may be located in the source code of the target application, and further, a bytecode adjustment statement is determined according to the key statement and the source code.

Illustratively, determining the bytecode adjustment statement based on the key statement and the source code may be: determining original log collection statements used for log collection in the source codes, and determining required log collection statements according to key statements; determining a bytecode adjustment statement based on the original log collection statement and the demand log collection statement.

The requirement log collection statement may be a statement for collecting connection information between the target application and the middleware in the running process. Each key statement for connecting the middleware may correspond to a requirement log collection statement to collect connection information of the target application and each middleware, that is, entry and exit information of the target application.

Specifically, determining the bytecode adjustment statement based on the original log collection statement and the required log collection statement may be: determining repeated statements in the original log collection statements and other statements except the required log collection statements as the log collection statements to be deleted, and generating byte code adjustment statements according to the log collection statements to be deleted; and/or determining other statements except the original log acquisition statement in the required log acquisition statement as the log acquisition statement to be added, and generating a bytecode adjustment statement according to the log acquisition statement to be added.

Locating the key statements and determining bytecode adjustment statements from the key statements and source code has the benefits of: the connection information between the application and the middleware is usually information needing attention, and the key statements connected with the middleware in the source code are positioned to generate byte code adjusting statements through the key statements, so that the target application can output key log information connected with the middleware, the information acquisition of the connection between the target application and the middleware is realized, such as the access information or the access information of the target application, and the subsequent problem troubleshooting and positioning of the target application are facilitated.

In this embodiment, besides the operation statement for executing the operation of deleting, modifying or adding the bytecode file, the bytecode adjustment statement may further include a basic statement for supporting editing of the bytecode file, or may further include a statement for calling another bytecode editing class library.

In a specific embodiment, determining the bytecode adjustment statement based on the key statement and the source code includes: determining a bytecode adjustment statement based on the key statement, the source code and the call statement for the bytecode editing class library.

Specifically, an operation statement may be generated according to the key statement and the source code, where the operation statement includes at least one of a delete statement, a modify statement, and an add statement; further, determining a bytecode adjustment statement according to the operation statement and a call statement aiming at the bytecode editing class library.

The operation statements are used for executing deletion, modification or addition operation on the byte code files, and the calling statements are used for calling the byte code editing class library so as to support the execution of the operation statements through the files in the byte code editing class library. For example, the calling statement may be a statement that calls a Log interface method that needs to be inserted at the location where the information is primarily collected.

In the embodiment, the bytecode adjustment statement is generated by the key statement, the source code and the calling statement aiming at the bytecode editing class library, so that the bytecode editing function of calling the bytecode editing class library by the calling statement is realized, further, the bytecode adjustment based on the bytecode editing class library is realized, the basic statement for supporting the bytecode file editing does not need to be separately configured in the log collection proxy file, the existing bytecode editing class library is directly called, and the generation efficiency of the log collection proxy file is improved.

Illustratively, the bytecode editing class library includes a language programming helper class library. For example the Javassist class library. Of course, the bytecode editing class library is not limited to the language programming helper class library, but may be a class library in a bytecode manipulation framework (e.g., java bytecode manipulation framework).

It should be noted that the advantage of calling the language programming helper class library is: compared with a bytecode manipulation framework, the language programming assistant class library is lighter, loading resources of the log collection agent file can be reduced by calling the language programming assistant class library, and the adjusting efficiency of the original bytecode file of the target application can be improved.

S120, based on the byte code adjusting statement, the original byte code file of the target application is accessed, and the original log collecting statement in the original byte code file is adjusted to obtain the target byte code file.

Specifically, after the log collection proxy file is loaded, the original bytecode file of the target application is accessed according to the bytecode adjustment statement in the log collection proxy file, and the original log collection statement in the original bytecode file is modified, deleted or inserted.

The original bytecode file may be an intermediate code file compiled according to a source code of the target application. Class files, which consist of byte codes (bytecodes) compiled from source code. The original log collection statement in the original bytecode file may be a code composed of bytecodes.

In a specific embodiment, the adjusting of the original log collection statement in the original bytecode file includes at least one of the following: deleting part of original log collection statements in an original byte code file based on deletion statements in the byte code adjustment statements; modifying part or all of original log collection statements in an original byte code file based on modification statements in the byte code adjustment statements; and determining a log acquisition statement to be added based on an addition statement in the byte code adjustment statement, and inserting the log acquisition statement to be added into the original byte code file.

The delete statement may be used to remove a part of the original log collection statement in the original bytecode file from the original bytecode file, for example, to delete a duplicate original log collection statement. The modification statement may be used to update a portion of the original log collection statement in the original bytecode file, e.g., modify the log storage path. The add statement may insert a log collection statement to be added to the original bytecode file.

Specifically, after the original log collection statement in the original bytecode file is adjusted, the adjusted original bytecode file can be used as the target bytecode file.

In the above embodiment, the deletion statement in the bytecode adjustment statement can delete the duplicate or useless log collection statement, thereby avoiding collecting a large amount of useless or duplicate log data, and solving the technical problem of editing the useless or duplicate log collection statement in the source code in the application development stage. The modification of the wrong log collection statement can be realized by adjusting the modification statement in the statement through the byte code, so that the problem that log data cannot be collected due to the error of the log collection statement is avoided, and the technical problem that the wrong log collection statement is edited in the source code in the application development stage is solved. By adjusting the added statements in the statements through the byte codes, the addition of the collected statements of the demand logs can be realized, the collection of service demand logs is further realized, and the technical problem that useful log statements are collected in source codes without being edited in the application development stage is solved.

And S130, acquiring target log data corresponding to the target log acquisition statement based on the target log acquisition statement in the target byte code file in the running process of the target application.

Specifically, after the original bytecode file is adjusted to obtain the target bytecode file, target log data in the running process of the target application can be collected according to the target log collection statements in the target bytecode file. The target log collection statement may be a code for collecting a log, which is formed by each bytecode in the target bytecode file.

For example, after obtaining the target bytecode file, the target bytecode file corresponding to the target application can be determined based on the target bytecode file. Furthermore, the target application can output target log data corresponding to the target log collection statement by itself by running the target code file.

Specifically, the target bytecode file can be interpreted as a target bytecode file by the virtual machine. For example, the object code file may be a machine code file. For example, when the virtual machine is started, the original bytecode file can be adjusted according to the log collection agent file, and the target bytecode file can be interpreted as a machine code file that can be run.

It should be noted that, the method provided in this embodiment adjusts the original bytecode file of the target application, does not affect the source code file of the target application, and solves the technical problem in the prior art that source codes are not reusable due to source code modification. In the method provided by this embodiment, if the target application needs to output other service requirement logs, the log collection proxy file may be configured again according to the service requirement, and the target application may be restarted to obtain the original bytecode file of the target application again, and then the original bytecode file is adjusted by the method provided by this embodiment to obtain the log data meeting the service requirement, so that the collection of the application logs is flexible. Moreover, if potential safety hazards exist, all directly-referenced and indirectly-referenced dependent packages need to be checked in the prior art, but the method provided by the embodiment can directly solve the problem of positioning through the collected application running logs, and the referenced dependent packages do not need to be checked.

According to the technical scheme of the embodiment, the log collection proxy file associated with the target application is loaded in response to the start of the target application, the original log collection statement in the original byte code file of the target application is regulated through the byte code regulation statement in the log collection proxy file, the target log collection statement capable of collecting target log data in the running process of the target application is obtained, and the log meeting the service requirement in the running process of the application is obtained through regulating the byte code of the application.

Fig. 2 is a schematic flow diagram of another application log collection method according to an embodiment of the present invention, and in this embodiment, a process of collecting logs in a process of adjusting an original bytecode file based on a bytecode adjustment statement according to a proxy log collection statement in a log collection proxy file is added on the basis of the above embodiment. As shown in fig. 2, the method includes:

s210, responding to the starting of the target application, and loading a log collection proxy file associated with the target application, wherein the log collection proxy file comprises a byte code adjusting statement and a proxy log collection statement.

S220, based on the bytecode adjustment statement, accessing the original bytecode file of the target application, and adjusting the original log acquisition statement in the original bytecode file to obtain the target bytecode file.

And S230, acquiring target log data corresponding to the target log acquisition statement based on the target log acquisition statement in the target byte code file in the running process of the target application.

S240, based on the proxy log collection statement, performing log collection on the process of adjusting the original byte code file based on the byte code adjustment statement to obtain proxy log data corresponding to the log collection proxy file.

The proxy log collection statement may be a statement in the log collection proxy file for collecting log data loaded by the log collection proxy file.

Specifically, after the log collection agent file is loaded, the log collection agent program written in the log collection agent file is operated, and the log collection agent program can adjust the original bytecode file of the target application according to the bytecode adjustment statement, so that the target application performs log collection according to the adjusted target bytecode file. In this embodiment, the proxy log collection statement may be used to collect an operation log of a log collection proxy program, that is, log collection is performed in a process of adjusting an original bytecode file based on a bytecode adjustment statement, so as to obtain proxy log data.

In this embodiment, a corresponding bytecode adjustment statement may be configured in advance according to a source code of the target application, and meanwhile, a statement for collecting log data in a bytecode adjustment process is configured.

In a specific implementation manner, the method provided in this embodiment further includes: acquiring a first statement for defining a storage path of the proxy log and a second statement for defining the level of the proxy log; determining a proxy log collection statement based on the first statement and the second statement.

That is, the configuration process for the proxy log collection statement may be divided into the following steps: firstly, defining a storage path of proxy log data to obtain a first statement; then, defining the log grade of the proxy log data to obtain a second statement; and finally, constructing a proxy log collection statement based on the first statement and the second statement.

Wherein, the log level of the proxy log data can represent the output priority of the log. For example, the proxy log level may be the general information most serious information (severe), the warning information (warning), (info), the configuration information (config), the friendly information (fine), the more friendly information (finder), the most friendly information (finest), and the like; the high-level log data can be preferentially output.

In the above embodiment, the first sentence and the second sentence may be generated by default, or the first sentence and/or the second sentence edited by the user may be acquired in response to an operation of the user on the terminal device. By acquiring the first statement and the second statement and determining the proxy log collection statement according to the first statement and the second statement, the log collection in the byte code adjustment process is realized, and the subsequent problem troubleshooting and positioning are facilitated.

In the above embodiment, the proxy log collection statement may further include, in addition to the first statement and the second statement, a basic statement that supports log collection, or may further include a statement that is used to call a log framework.

In a specific embodiment, determining a proxy log collection statement based on a first statement and a second statement comprises: determining a proxy log collection statement based on the first statement, the second statement, and the call statement for the log frame.

The call statement for the log frame may be a log implementation class in the call log frame. Specifically, the call statement, the first statement, and the second statement may be combined to obtain the proxy log collection statement.

Based on the call statements to the log framework, the benefit of determining proxy log collection statements is: the method and the device can realize log acquisition based on the existing log frame, do not need to configure basic statements supporting log realization in the log acquisition proxy file independently, and improve the generation efficiency of the log acquisition proxy file.

Illustratively, the logging framework comprises a system native logging framework. For example, the system native Logging framework may be a JUL (Java Logging) framework. The benefits of using a native logging framework are: the log collection can be realized by using the self-contained class library without introducing other redundant class libraries, so that the generation efficiency of the log collection proxy file is improved, and the loading resources of the log collection proxy file can be reduced.

Of course, in addition to the first statement and the second statement, a third statement for defining the size limit of the proxy log and a fourth statement for defining the output format of the proxy log may be obtained; further, a proxy log collection statement is formed based on the first statement, the second statement, the third statement and the fourth statement.

It should be further noted that, in this embodiment, the execution sequence of S230 and S240 is not sequential, and may be that, in the process of adjusting the original bytecode file based on the bytecode adjustment statement, proxy log data is acquired based on the proxy log acquisition statement, and after the original bytecode file is adjusted based on the bytecode adjustment statement, target bytecode data is acquired based on the target bytecode file obtained through adjustment.

According to the technical scheme of the embodiment, the original byte code file of the target application is adjusted based on the byte code adjusting statement in the log collection proxy file, so that the target application outputs the target log data, meanwhile, the log collection is performed aiming at the adjusting process of the original byte code file based on the proxy log collection statement in the log collection proxy file, so that the proxy log data related to the log collection proxy file is obtained, the log collection aiming at the log collection proxy file is realized, and the follow-up problem troubleshooting and positioning are facilitated.

Fig. 3A is a schematic flow chart of another application log collection method according to an embodiment of the present invention, and in this embodiment, a process of performing associated storage on target log data and/or proxy log data according to a log transmission statement in a log collection proxy file is added on the basis of the above embodiment. As shown in fig. 3A, the method includes:

s310, responding to the starting of the target application, and loading a log collection proxy file associated with the target application, wherein the log collection proxy file comprises byte code adjusting statements, proxy log collection statements and log transmission statements.

S320, based on the byte code adjusting statement, accessing the original byte code file of the target application, and adjusting the original log collecting statement in the original byte code file to obtain the target byte code file.

S330, collecting target log data corresponding to the target log collection statement in the running process of the target application based on the target log collection statement in the target byte code file.

S340, based on the proxy log collection statement, performing log collection on the process of adjusting the original byte code file based on the byte code adjustment statement to obtain proxy log data corresponding to the log collection proxy file.

S350, generating a log identifier corresponding to the current log data to be transmitted based on the log transmission statement, and sending the current log data to be transmitted and the log identifier to an index database for associated storage; the current log data to be transmitted comprises target log data and/or proxy log data.

In this embodiment, after the target log data and the proxy log data are acquired, the target log data and/or the proxy log data may be sent to the index database for storage according to the log transmission statement in the log acquisition proxy file. Wherein the index database may be an Elasticsearch database.

Specifically, according to the log transmission statement, a log identifier corresponding to the current log data to be transmitted is generated for the current log data to be transmitted, and then the current log data to be transmitted and the log identifier are stored in the index database in an associated manner.

For example, generating the log identifier corresponding to the log data to be currently transmitted may be: according to the data source corresponding to the current log data to be transmitted, generating log identifiers corresponding to the current log data to be transmitted, so that the log identifiers of the log data with the same data source are the same, and the log identifiers of the log data with different data sources are different.

The data sources are the same, and the target applications are the same, and the clients requesting the target applications are the same. For example, a client identifier corresponding to the current client may be determined in a thread local variable manner, and then the client identifier is used as a log identifier corresponding to target log data generated based on the current client.

Referring to fig. 3B, a schematic diagram of a log collection process is shown. When a plurality of clients concurrently request to access a target application, for example, a target application may be acquired for each client, and further, for the target log data corresponding to each client, log identifiers may be generated, and the target log data and the corresponding log identifiers are stored in an index database in an associated manner, so as to ensure the consistency of the traffic of the same client, so that the identifiers of the traffic of the same client are always the same, and at the same time, the traffic of each client cannot be distinguished.

Considering that when the data volume of the current log data to be transmitted is large, the operation of writing into the index database at high concurrency may cause the pressure of the index database to be too large, therefore, the message queue may be used for buffering the writing operation at high concurrency to the index database.

In a specific embodiment, sending the current log data to be transmitted and the log identifier to an index database for associated storage includes: and if the log data volume of the current log data to be transmitted exceeds a preset threshold value, sending the current log data to be transmitted and the log identification to an index database reading area in a preset message queue corresponding to the index database, so that the index database acquires the current log data to be transmitted and the log identification from the index database reading area.

The preset message queue may be understood as a middleware between the target application and the index database, or a middleware between the log collection agent file and the index database. Illustratively, the pre-set message queue may be kafka middleware, a rabbitmessage queue, a Zero message queue, or a Mete message queue.

Specifically, when the log data amount exceeds a preset threshold, the current log data to be transmitted and the log identifier may be sent to a preset message queue, so that the preset message queue caches the current log data to be transmitted and the log identifier to an index library reading area, and further, the index database may read the current log data to be transmitted and the log identifier from the index library reading area.

The log data are cached in the preset message queue, so that the condition that the write operation to the index database at high concurrency cannot be processed is avoided, and the log data can be written into the index database.

After the target log data and/or the proxy log data are written into the index database, the front end can acquire the log data stored in the index database and display the log data by sending a log data viewing request.

Exemplarily, as shown in fig. 3C, a schematic diagram of an application log processing process is shown, where the collection of the target log data of each target application is implemented by loading a log collection agent file associated with each target application, and further, the target log data of each target application is sent to a preset message queue, and the target log data is stored in an index database through the preset message queue; further, the front end may request the target log data from the index database through an interface provided by the index database, and display the obtained target log data.

According to the technical scheme, the log identification corresponding to the current log data to be transmitted is generated according to the log transmission statement in the log collection agent file, and then the current log data to be transmitted and the log identification are sent to the index database for associated storage, so that the sources of the log data in the index database can be conveniently distinguished according to the identification, meanwhile, the identification of the log data with the same source in the index database can be ensured to be consistent, and the problem troubleshooting and the positioning are facilitated.

Fig. 4 is a schematic structural diagram of an application log collecting device according to an embodiment of the present invention. As shown in fig. 4, the apparatus includes a proxy file loading module 410, a bytecode adjustment module 420, and a log collection module 430.

The proxy file loading module 410 is configured to, in response to the start of the target application, load a log collection proxy file associated with the target application, where the log collection proxy file includes a bytecode adjustment statement;

the bytecode adjusting module 420 is configured to access an original bytecode file of the target application based on the bytecode adjustment statement, and adjust an original log acquisition statement in the original bytecode file to obtain a target bytecode file;

the log collection module 430 is configured to collect, based on a target log collection statement in the target bytecode file, target log data corresponding to the target log collection statement during an operation process of the target application.

According to the technical scheme of the embodiment, the log collection proxy file associated with the target application is loaded together in response to the start of the target application through the proxy file loading module, the original log collection statement in the original byte code file of the target application is regulated through the byte code regulation module based on the byte code regulation statement in the log collection proxy file to obtain the target log collection statement capable of collecting target log data in the running process of the target application, and then the log collection is carried out through the log collection module based on the target byte code file to obtain the log meeting the service requirement in the running process of the application through regulating the byte code of the application.

On the basis of the embodiment, the application log acquisition device also comprises an agent acquisition module; and the proxy acquisition module is used for acquiring logs in the process of adjusting the original byte code file based on the byte code adjusting statement based on the proxy log acquisition statement to obtain proxy log data corresponding to the log acquisition proxy file.

On the basis of the embodiment, the application log acquisition device further comprises a log transmission module, wherein the log transmission module is used for generating a log identifier corresponding to the current log data to be transmitted based on the log transmission statement, and sending the current log data to be transmitted and the log identifier to an index database for associated storage; the current log data to be transmitted comprises the target log data and/or the proxy log data.

On the basis of the above embodiment, the log transmission module is further configured to send the log data to be transmitted and the log identifier to an index base reading area in a preset message queue corresponding to the index database if it is detected that the log data amount of the log data to be transmitted currently exceeds a preset threshold, so that the index database obtains the log data to be transmitted currently and the log identifier from the index base reading area.

On the basis of the embodiment, the application log acquisition device also comprises an adjusting statement generating module; and the adjusting statement generating module is used for determining a key statement used for connecting the middleware corresponding to the target application in the source code of the target application and determining the bytecode adjusting statement based on the key statement and the source code.

On the basis of the embodiment, the application log acquisition device also comprises an agent statement generation module; the agent statement generation module is used for acquiring a first statement used for defining an agent log storage path and a second statement used for defining the agent log level; determining the proxy log collection statement based on the first statement and the second statement.

On the basis of the above embodiment, the adjusting statement generating module is further configured to determine the bytecode adjusting statement based on the key statement, the source code, and the call statement for the bytecode editing class library.

On the basis of the above embodiment, the proxy statement generation module is further configured to determine the proxy log collection statement based on the first statement, the second statement, and a call statement for a log frame.

On the basis of the above embodiment, the bytecode adjustment module 420 is further configured to perform at least one of the following operations:

deleting part of original log collection statements in the original byte code file based on deletion statements in the byte code adjusting statements; modifying part or all of original log collection statements in the original bytecode file based on modification statements in the bytecode adjustment statements; determining a log collection statement to be added based on an addition statement in the bytecode adjustment statement, and inserting the log collection statement to be added into the original bytecode file.

On the basis of the above embodiment, the bytecode editing class library includes a language programming helper class library.

On the basis of the above embodiment, the logging framework comprises a system native logging framework.

The application log acquisition device provided by the embodiment of the invention can execute the application log acquisition method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 may also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to the bus 14.

A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as applying a log collection method.

In some embodiments, the application log collection method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the application log collection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the application log collection method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the application log collection method of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

An embodiment of the present invention further provides a computer-readable storage medium, where a computer instruction is stored, where the computer instruction is used to enable a processor to execute a method for collecting an application log, where the method includes:

responding to the starting of a target application, and loading a log collection proxy file associated with the target application, wherein the log collection proxy file comprises a byte code adjusting statement;

accessing an original bytecode file of the target application based on the bytecode adjustment statement, and adjusting an original log acquisition statement in the original bytecode file to obtain a target bytecode file;

and acquiring target log data corresponding to the target log acquisition statement in the running process of the target application based on the target log acquisition statement in the target byte code file.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (12)

1. An application log collection method, comprising:

responding to the starting of a target application, and loading a log collection proxy file associated with the target application, wherein the log collection proxy file comprises a byte code adjusting statement;

accessing an original bytecode file of the target application based on the bytecode adjustment statement, and adjusting an original log acquisition statement in the original bytecode file to obtain a target bytecode file;

and acquiring target log data corresponding to the target log acquisition statement in the running process of the target application based on the target log acquisition statement in the target byte code file.

2. The method of claim 1, wherein the log collection agent file further comprises an agent log collection statement, and wherein the method further comprises:

and based on the proxy log collection statement, performing log collection on the process of regulating the original byte code file based on the byte code regulation statement to obtain proxy log data corresponding to the log collection proxy file.

3. The method of claim 2, wherein the log collection agent file further comprises log transfer statements, and the method further comprises:

generating a log identifier corresponding to the current log data to be transmitted based on the log transmission statement, and sending the current log data to be transmitted and the log identifier to an index database for associated storage;

the current log data to be transmitted comprises the target log data and/or the proxy log data.

4. The method according to claim 3, wherein the sending the log data to be transmitted currently and the log identifier to an index database for associated storage comprises:

and if the log data volume of the current log data to be transmitted exceeds a preset threshold value, sending the current log data to be transmitted and the log identification to an index database reading area in a preset message queue corresponding to the index database, so that the index database acquires the current log data to be transmitted and the log identification from the index database reading area.

5. The method of claim 1, further comprising:

determining a key statement for connecting the middleware corresponding to the target application in the source code of the target application;

determining the bytecode adjustment statement based on the key statement and the source code.

6. The method of claim 2, further comprising:

acquiring a first statement for defining a storage path of the proxy log and a second statement for defining the level of the proxy log;

determining the proxy log collection statement based on the first statement and the second statement.

7. The method of claim 5, wherein determining the bytecode adjustment statement based on the key statement and the source code comprises:

and determining the bytecode adjusting statement based on the key statement, the source code and the calling statement aiming at the bytecode editing class library.

8. The method of claim 6, wherein the determining the proxy log collection statement based on the first statement and the second statement comprises:

determining the proxy log collection statement based on the first statement, the second statement, and a call statement for a log frame.

9. The method of claim 1, wherein the adjusting the raw log collection statement in the raw bytecode file comprises at least one of:

based on a deletion statement in the bytecode adjustment statement, deleting part of original log collection statements in the original bytecode file;

modifying part or all of original log collection statements in the original bytecode file based on modification statements in the bytecode adjustment statements;

determining a log collection statement to be added based on an addition statement in the bytecode adjustment statement, and inserting the log collection statement to be added into the original bytecode file.

10. An application log collection apparatus, comprising:

the system comprises a proxy file loading module, a log collection proxy file loading module and a data processing module, wherein the proxy file loading module is used for responding to the starting of a target application and loading the log collection proxy file associated with the target application, and the log collection proxy file comprises a byte code adjusting statement;

the bytecode adjusting module is used for accessing the original bytecode file of the target application based on the bytecode adjusting statement and adjusting the original log acquisition statement in the original bytecode file to obtain a target bytecode file;

and the log acquisition module is used for acquiring target log data corresponding to the target log acquisition statement in the running process of the target application based on the target log acquisition statement in the target byte code file.

11. An electronic device, characterized in that the electronic device comprises:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the application log collection method of any one of claims 1-9.

12. A computer-readable storage medium having stored thereon computer instructions for causing a processor to implement the application log collection method of any one of claims 1-9 when executed.

Images