CN113111137A - Distributed system log real-time display method, device, equipment and storage medium - Google Patents
Distributed system log real-time display method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113111137A CN113111137A CN202110483811.1A CN202110483811A CN113111137A CN 113111137 A CN113111137 A CN 113111137A CN 202110483811 A CN202110483811 A CN 202110483811A CN 113111137 A CN113111137 A CN 113111137A
- Authority
- CN
- China
- Prior art keywords
- logs
- queue
- log
- background
- distributed system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004458 analytical method Methods 0.000 claims description 7
- 230000001174 ascending effect Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 10
- 238000007726 management method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 238000012856 packing Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 241000555745 Sciuridae Species 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013481 data capture Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
- G06F16/287—Visualization; Browsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a data display technology, and discloses a distributed system log real-time display method, which comprises the following steps: establishing a first buffer queue and a second buffer queue; the method comprises the steps of obtaining system logs of a plurality of background systems in a distributed system, and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs; writing the encoding logs into the first cache queue and the second cache queue in sequence; and when the queue length occupied by the coding logs in the second cache queue is greater than a preset queue threshold value, writing the coding logs in the first cache queue into a preset display page in real time for log display. In addition, the invention also relates to a block chain technology, and the system log can be stored in a node of the block chain. The invention also provides a distributed system log real-time display device, electronic equipment and a computer readable storage medium. The invention can solve the problem that logs of a plurality of background services in a distributed system cannot be displayed in real time.
Description
Technical Field
The invention relates to the technical field of data display, in particular to a distributed system log real-time display method and device, electronic equipment and a computer readable storage medium.
Background
With the enhancement of functions and the explosive growth of data in the existing computer systems, some functions need to be completed by huge computer groups together, so that the operation and maintenance costs of the centralized large computer groups are increased, and the concept of distributed clustering is developed.
The operation and maintenance cost of a huge computer group can be greatly reduced by the distributed cluster, but because the background services supported by different computers in the distributed cluster are far away in distribution distance, logs generated by each background service in the operation and maintenance process are mostly stored independently, so that the efficiency of the distributed system in operation and maintenance is low through the logs, and therefore how to realize real-time display of the logs of each background service in the distributed system becomes a problem to be solved urgently.
Disclosure of Invention
The invention provides a method and a device for displaying logs of a distributed system in real time and a computer readable storage medium, and mainly aims to solve the problem that logs of a plurality of background services in the distributed system cannot be displayed in real time.
In order to achieve the above object, the present invention provides a distributed system log real-time display method, which includes:
establishing a first buffer queue and a second buffer queue;
the method comprises the steps of obtaining system logs of a plurality of background systems in a distributed system, and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs;
writing the encoding logs into the first cache queue and the second cache queue in sequence;
and when the queue length occupied by the coding logs in the second cache queue is greater than a preset queue threshold value, writing the coding logs in the first cache queue into a preset display page in real time for log display.
Optionally, the establishing the first buffer queue and the second buffer queue includes:
creating a blank data queue with a preset length in a preset database;
determining a cache data type;
and performing queue variable assignment on the blank data table according to the cache data type to obtain a first cache queue and a second cache queue.
Optionally, the obtaining the system log of the plurality of backend systems in the distributed system includes:
acquiring log generation requests corresponding to a plurality of background systems in a distributed system;
performing ip address analysis on the log capture request to obtain background ip addresses corresponding to a plurality of background systems;
and synchronously capturing the system logs of the background systems by using the background ip addresses.
Optionally, the performing ip address resolution on the log capture request to obtain background ip addresses corresponding to a plurality of background systems includes:
traversing the log generation request and determining the position of a field separator in the log generation request;
splitting the fields of the log generation request according to the positions of the field separators to obtain a plurality of split fields;
carrying out serial number labeling on the plurality of split fields according to a split sequence;
selecting a splitting field with a preset sequence number as the target data field;
and carrying out field analysis on the target data field to obtain background ip addresses corresponding to a plurality of background systems.
Optionally, the performing unique index coding on the system log of each background system to obtain a coded log includes:
respectively acquiring system parameters of each background system;
generating a unique identifier corresponding to each background system according to the system parameters;
generating an ordered code and writing the unique identifier into the ordered code to obtain a unique code;
and marking the system log of each background system by using the unique code to obtain a code log.
Optionally, the sequentially writing the encoded logs into the first buffer queue and the second buffer queue includes:
writing the encoding log into a first cache queue;
if the queue length occupied by the coding logs in the first cache queue is smaller than the preset length, continuing to write the coding logs into the first cache queue;
and if the queue length occupied by the coding logs in the first cache queue is greater than or equal to the preset length, writing the coding logs into a second cache queue.
Optionally, the writing, in real time, the encoding log in the first cache queue into a display page for log display by using a data link includes:
classifying the code logs in the first cache queue according to the unique codes to obtain display logs corresponding to the subsystems;
arranging the display logs in an ascending order according to the unique codes to obtain a log display sequence;
and writing the log display sequence into a preset display page for log display.
In order to solve the above problem, the present invention further provides a distributed system log real-time display device, including:
the queue creating module is used for creating a first buffer queue and a second buffer queue;
the log coding module is used for acquiring system logs of a plurality of background systems in the distributed system and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs;
the log caching module is used for sequentially writing the encoded logs into the first caching queue and the second caching queue;
and the log display module is used for writing the coded logs in the first cache queue into a preset display page in real time for log display when the queue length occupied by the coded logs in the second cache queue is greater than a preset queue threshold value.
In order to solve the above problem, the present invention also provides an electronic device, including:
a memory storing at least one instruction; and
and the processor executes the instructions stored in the memory to realize the distributed system log real-time display method.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, where at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in an electronic device to implement the above method for displaying a distributed system log in real time.
The system logs of each background system are respectively marked by the unique codes to obtain the coded logs, so that the disordered writing of the background system logs can be avoided, and the efficiency and regularity of subsequent log display can be improved; the method comprises the steps of establishing a first cache queue and a second cache queue, sequentially utilizing different cache queues to collect logs of a plurality of background services in the distributed system, writing the logs in the cache queues into a display page in real time for displaying, and effectively preventing the condition that when the display page is written again after all the coded logs in one cache queue are written into the display page, the written logs are null values, so that the log displaying is interrupted, the continuity of writing the coded logs is improved, and the real-time displaying of the logs of the plurality of background services in the distributed system is realized. Therefore, the method, the device, the electronic equipment and the computer readable storage medium for displaying the logs of the distributed system in real time can solve the problem that the logs of a plurality of background services in the distributed system cannot be displayed in real time.
Drawings
Fig. 1 is a schematic flowchart of a method for displaying logs of a distributed system in real time according to an embodiment of the present invention;
fig. 2 is a detailed flowchart of one step in the distributed system log real-time display method according to an embodiment of the present invention;
fig. 3 is a functional block diagram of a distributed system log real-time display apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device for implementing the distributed system log real-time display method according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the application provides a distributed system log real-time display method. The executing body of the distributed system log real-time showing method includes, but is not limited to, at least one of electronic devices that can be configured to execute the method provided by the embodiment of the present application, such as a server, a terminal, and the like. In other words, the distributed system log real-time presentation method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Fig. 1 is a schematic flow chart of a method for displaying a distributed system log in real time according to an embodiment of the present invention. In this embodiment, the method for displaying the distributed system log in real time includes:
and S1, establishing a first buffer queue and a second buffer queue.
In an actual application scenario of the present invention, the first buffer queue and the second buffer queue may be data buffer queues with a preset length, so as to buffer data with a preset length, and the preset lengths of the first buffer queue and the second buffer queue may be the same or different.
In the embodiment of the invention, the first cache queue and the second cache queue can be created in a database with a data cache function, such as Redis, memcached and the like, through a database instruction. For example, a blank data queue with a preset length is created in a preset database through a database instruction, and parameter assignment is performed on the created blank data queue according to the type of data to be cached, so that creation of the first cache queue and the second cache queue is realized.
In the embodiment of the present invention, the cache data type includes, but is not limited to, data of types such as String, hash, map, set, and sortset, and the cache data type is determined by a data type included in a log of the distributed system, for example, if the log of the distributed system includes String type data, the cache data type is String type data. The cache data type may be a plurality of different types of data at the same time.
Further, in the embodiment of the present invention, a pre-stored queue variable value table is obtained, and queue variable assignment is performed on the blank data table by using the queue variable value table, so as to obtain the first buffer queue and the second buffer queue. The queue variable value table can be preset by a user, and the queue variable corresponding to various different cache data types is contained in the variable value table. Wherein the queue variables include, but are not limited to, primary key value type, and the like.
In an embodiment of the present invention, referring to fig. 2, the establishing the first buffer queue and the second buffer queue includes:
s21, creating a blank data queue with a preset length in a preset database;
s22, determining the type of the cache data;
and S23, performing queue variable assignment on the blank data table according to the cache data type to obtain a first cache queue and a second cache queue.
The embodiment of the invention can realize the alternate writing of the data by establishing the first buffer queue and the second buffer queue, for example, the data is firstly written into the first buffer queue, and when the data amount in the first buffer queue reaches the preset length, the data is written into the second buffer queue, thereby avoiding the interruption of the log writing and improving the efficiency and the real-time performance of obtaining the log.
S2, obtaining system logs of a plurality of background systems in the distributed system, and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs.
In an embodiment of the present invention, the obtaining system logs of a plurality of background systems in a distributed system includes:
acquiring log generation requests corresponding to a plurality of background systems in a distributed system;
performing ip address analysis on the log capture request to obtain background ip addresses corresponding to a plurality of background systems;
and synchronously capturing the system logs of the background systems by using the background ip addresses.
In detail, the log generation request is a data request automatically generated by a background system of the distributed system when generating the system log. According to the embodiment of the invention, the ASM enhanced byte code filter is used for acquiring the log generation requests corresponding to a plurality of background systems in the distributed system, the ASM enhanced byte code filter is a code analysis tool based on a java byte code level, and the ASM enhanced byte code filter is used for monitoring and acquiring the log generation requests, so that the success rate of acquiring the log generation requests can be improved.
Specifically, the ip address resolution of the log capture request to obtain the background ip addresses corresponding to the multiple background systems includes:
traversing the log generation request and determining the position of a field separator in the log generation request;
splitting the fields of the log generation request according to the positions of the field separators to obtain a plurality of split fields;
carrying out serial number labeling on the plurality of split fields according to a split sequence;
selecting a splitting field with a preset sequence number as the target data field;
and carrying out field analysis on the target data field to obtain background ip addresses corresponding to a plurality of background systems.
For example, if the data packing request is "qwe < rt < yuio < pl", where < is a field delimiter, the field splitting of the data packing request according to the position of the field delimiter in the data packing request is: the method comprises the following steps of dividing four division fields of 'qwe', 'rt', 'yuio' and 'pl', and marking sequence numbers of the division fields as follows according to a division sequence: 1 "qwe", 2 "rt", 3 "yuio" and 4 "pl", for example, the preset sequence number is 2, the field "rt" is determined to be the target data field.
Generally, the position of each field in the log generation request is fixed, so that the target field for identifying the background ip address corresponding to the background system can be obtained through screening of the preset sequence number.
Further, in the embodiment of the present invention, a parser may be used to perform field parsing on the target data field to obtain background ip addresses corresponding to multiple background systems, where the parser includes carakan C/C + +, squirrel fisherc + +, squirrel fisherxtremc + +, and the like.
The embodiment of the invention acquires the system logs of the plurality of background systems according to the background ip addresses corresponding to the plurality of background systems in a multithreading parallel mode, can avoid acquiring the system logs one by one according to the ip fields, and is favorable for improving the efficiency of acquiring the system logs.
In detail, the uniquely indexing and encoding the system log of each background system to obtain an encoded log respectively includes:
respectively acquiring system parameters of each background system;
generating a unique identifier corresponding to each background system according to the system parameters;
generating an ordered code and writing the unique identifier into the ordered code to obtain a unique code;
and marking the system log of each background system by using the unique code to obtain a code log.
In detail, the embodiment of the invention can capture the system parameters of each background system from each background system by using a point python statement with a data capture function, wherein the system parameters are used for uniquely identifying the background systems, namely the system parameters of each background system are inconsistent with the system parameters of other background systems.
The embodiment of the invention can adopt the PrimaryKeyService class in the Redis database to generate the unique increasing serial number, and the unique increasing serial number is 1, 2, 3, 4, 5 … … for example
For example: the method comprises a background system A and a background system B, wherein the system parameter of the background system A is a, and the system parameter of the background system B is B. In the embodiment of the present invention, the system parameters of the background system a and the system parameters of the background system B are written into the generated background system a and the generated background system B, respectively, and the unique code of the background system a is obtained as follows: 1a, 2a, 3a, 4a, 5a … …, the unique code of backend system B is 1B, 2B, 3B, 4B, 5B … …
The embodiment of the invention respectively marks the system logs of each background system by using the unique codes to obtain the coded logs, can avoid the disordered writing of a plurality of background system logs, and is favorable for improving the efficiency and regularity of subsequent log display.
And S3, writing the encoding log into the first buffer queue and the second buffer queue in sequence.
In this embodiment of the present invention, the sequentially writing the encoded logs into the first buffer queue and the second buffer queue includes:
writing the encoding log into a first cache queue;
if the queue length occupied by the coding logs in the first cache queue is smaller than the preset length, continuing to write the coding logs into the first cache queue;
and if the queue length occupied by the coding logs in the first cache queue is greater than or equal to the preset length, writing the coding logs into a second cache queue.
Specifically, each encoded log occupies one queue length.
In detail, in the embodiment of the present invention, the encoded log is preferentially written into the first cache queue, when the queue length occupied by the encoded log in the first cache queue is smaller than the preset length, the encoded log is continuously written into the first cache queue, and when the queue length occupied by the encoded log in the first cache queue is greater than or equal to the preset length, the data is written into the second cache queue, thereby avoiding interruption during log writing, and improving efficiency and real-time performance of obtaining the log.
And S4, when the queue length occupied by the coding logs in the second cache queue is larger than a preset queue threshold value, writing the coding logs in the first cache queue into a preset display page in real time for log display.
In the embodiment of the invention, whether the queue length occupied by the coding logs in the second cache queue is greater than a preset queue threshold value or not can be judged through a preset binary algorithm.
In detail, the binary algorithm may be:
g is a queue length occupied by the encoded logs in the second cache queue, α is a preset weight coefficient, k is the preset length, f (x) is a judgment value, when f (x) is 1, the queue length occupied by the encoded logs in the second cache queue is greater than a preset queue threshold, and when f (x) is 0, the queue length occupied by the encoded logs in the second cache queue is less than or equal to the preset queue threshold.
Further, in the embodiment of the present invention, when the queue length occupied by the encoded log in the second cache queue is greater than a preset queue threshold, the encoded log in the second cache queue may be written into the display page in real time through a pre-established data link for display. Similarly, when the queue length occupied by the coding logs in the first cache queue is greater than the queue threshold, the coding logs in the first cache queue can be written into the display page in real time through a pre-constructed data link for display.
When the length of the queue occupied by the coding log is larger than the queue threshold, the coding log is written into the display page in real time for log display, so that the log quantity stored in the queue can be effectively prevented from being overlarge, and the implementation of writing can ensure the implementation and display of the coding log on the display page.
In detail, the writing, in real time, the encoding log in the first cache queue into a presentation page for log presentation by using a data link includes:
classifying the code logs in the first cache queue according to the unique codes to obtain display logs corresponding to the subsystems;
arranging the display logs in an ascending order according to the unique codes to obtain a log display sequence;
and writing the log display sequence into a preset display page for log display.
For example, the system file includes standard log data of a background system a and standard log data of a background system B, the standard log data including the unique identifier a of the background system a in the unique code is collected as a display log of the background system a, and the standard log data including the unique identifier B of the background system B in the unique code is collected as a display log of the background system B.
Further, the display logs are arranged in an ascending order according to the unique codes, for example, the unique codes of the display logs of the background system a include: 1a, 3a, 2a, 4a and 5a, the unique codes of the display logs of the background system B include 1B, 3B, 2B, 4B and 5B, then the display logs of the background system a are arranged in an ascending order to obtain 1a, 2a, 3a, 4a and 5a, and the display logs of the background system B are arranged in an ascending order to obtain 1B, 2B, 3B, 4B and 5B.
In the embodiment of the invention, the display page is a preset web page, the log display sequence is written into the display page through a WEBSOCKET link for log display, and the WEBSOCKET link is a data transmission connection capable of realizing input and write into the web page.
The embodiment of the invention can effectively prevent the interruption of log display caused by the condition that the written-in value is null when the display page is written again after the coding logs in a certain cache queue are all written into the display page by setting at least two cache queues, thereby improving the continuity of the writing of the coding logs.
The system logs of each background system are respectively marked by the unique codes to obtain the coded logs, so that the disordered writing of the background system logs can be avoided, and the efficiency and regularity of subsequent log display can be improved; the method comprises the steps of establishing a first cache queue and a second cache queue, sequentially utilizing different cache queues to collect logs of a plurality of background services in the distributed system, writing the logs in the cache queues into a display page in real time for displaying, and effectively preventing the condition that when the display page is written again after all the coded logs in one cache queue are written into the display page, the written logs are null values, so that the log displaying is interrupted, the continuity of writing the coded logs is improved, and the real-time displaying of the logs of the plurality of background services in the distributed system is realized. Therefore, the method for displaying the logs of the distributed system in real time can solve the problem that the logs of a plurality of background services in the distributed system cannot be displayed in real time.
Fig. 3 is a functional block diagram of a distributed system log real-time display apparatus according to an embodiment of the present invention.
The distributed system log real-time display device 100 of the present invention may be installed in an electronic device. According to the implemented functions, the distributed system log real-time presentation apparatus 100 may include a queue creation module 101, a log encoding module 102, a log caching module 103, and a log presentation module 104. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions regarding the respective modules/units are as follows:
the queue creating module 101 is configured to create a first buffer queue and a second buffer queue;
the log coding module 102 is configured to obtain system logs of a plurality of background systems in the distributed system, and perform unique index coding on the system log of each background system to obtain a coded log;
the log caching module 103 is configured to sequentially write the encoded logs into the first cache queue and the second cache queue;
the log display module 104 is configured to, when the queue length occupied by the encoded logs in the second cache queue is greater than a preset queue threshold, write the encoded logs in the first cache queue into a preset display page in real time to perform log display.
In detail, when the modules in the distributed system log real-time display 100 according to the embodiment of the present invention are used, the same technical means as the distributed system log real-time display method described in fig. 1 and fig. 2 are adopted, and the same technical effect can be produced, which is not described herein again.
The system logs of each background system are respectively marked by the unique codes to obtain the coded logs, so that the disordered writing of the background system logs can be avoided, and the efficiency and regularity of subsequent log display can be improved; the method comprises the steps of establishing a first cache queue and a second cache queue, sequentially utilizing different cache queues to collect logs of a plurality of background services in the distributed system, writing the logs in the cache queues into a display page in real time for displaying, and effectively preventing the condition that when the display page is written again after all the coded logs in one cache queue are written into the display page, the written logs are null values, so that the log displaying is interrupted, the continuity of writing the coded logs is improved, and the real-time displaying of the logs of the plurality of background services in the distributed system is realized. Therefore, the device for displaying the logs of the distributed system in real time can solve the problem that the logs of a plurality of background services in the distributed system cannot be displayed in real time.
Fig. 4 is a schematic structural diagram of an electronic device for implementing a distributed system log real-time display method according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11 and a bus, and may further comprise a computer program, such as a distributed system log real-time presentation program 12, stored in the memory 11 and executable on the processor 10.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used to store not only application software installed in the electronic device 1 and various types of data, such as codes of the distributed system log real-time presentation program 12, but also temporarily store data that has been output or will be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (e.g., distributed system log real-time presentation programs, etc.) stored in the memory 11 and calling data stored in the memory 11.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
Fig. 4 only shows an electronic device with components, and it will be understood by those skilled in the art that the structure shown in fig. 4 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The distributed system log real-time presentation program 12 stored in the memory 11 of the electronic device 1 is a combination of a plurality of instructions, and when running in the processor 10, can realize:
establishing a first buffer queue and a second buffer queue;
the method comprises the steps of obtaining system logs of a plurality of background systems in a distributed system, and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs;
writing the encoding logs into the first cache queue and the second cache queue in sequence;
and when the queue length occupied by the coding logs in the second cache queue is greater than a preset queue threshold value, writing the coding logs in the first cache queue into a preset display page in real time for log display.
Specifically, the specific implementation method of the processor 10 for the instruction may refer to the description of the relevant steps in the embodiment corresponding to fig. 1, which is not described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:
establishing a first buffer queue and a second buffer queue;
the method comprises the steps of obtaining system logs of a plurality of background systems in a distributed system, and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs;
writing the encoding logs into the first cache queue and the second cache queue in sequence;
and when the queue length occupied by the coding logs in the second cache queue is greater than a preset queue threshold value, writing the coding logs in the first cache queue into a preset display page in real time for log display.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A distributed system log real-time display method is characterized by comprising the following steps:
establishing a first buffer queue and a second buffer queue;
the method comprises the steps of obtaining system logs of a plurality of background systems in a distributed system, and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs;
writing the encoding logs into the first cache queue and the second cache queue in sequence;
and when the queue length occupied by the coding logs in the second cache queue is greater than a preset queue threshold value, writing the coding logs in the first cache queue into a preset display page in real time for log display.
2. The method for displaying logs of a distributed system in real time according to claim 1, wherein the establishing of the first buffer queue and the second buffer queue comprises:
creating a blank data queue with a preset length in a preset database;
determining a cache data type;
and performing queue variable assignment on the blank data table according to the cache data type to obtain a first cache queue and a second cache queue.
3. The method for displaying the logs of the distributed system in real time according to claim 1, wherein the obtaining the logs of the multiple background systems in the distributed system comprises:
acquiring log generation requests corresponding to a plurality of background systems in a distributed system;
performing ip address analysis on the log capture request to obtain background ip addresses corresponding to a plurality of background systems;
and synchronously capturing the system logs of the background systems by using the background ip addresses.
4. The method for displaying logs of a distributed system in real time according to claim 3, wherein the step of performing ip address resolution on the log capture request to obtain background ip addresses corresponding to a plurality of background systems comprises the steps of:
traversing the log generation request and determining the position of a field separator in the log generation request;
splitting the fields of the log generation request according to the positions of the field separators to obtain a plurality of split fields;
carrying out serial number labeling on the plurality of split fields according to a split sequence;
selecting a splitting field with a preset sequence number as the target data field;
and carrying out field analysis on the target data field to obtain background ip addresses corresponding to a plurality of background systems.
5. The method for displaying the distributed system logs in real time according to claim 1, wherein the step of performing unique index coding on the system logs of each background system to obtain coded logs comprises the steps of:
respectively acquiring system parameters of each background system;
generating a unique identifier corresponding to each background system according to the system parameters;
generating an ordered code and writing the unique identifier into the ordered code to obtain a unique code;
and marking the system log of each background system by using the unique code to obtain a code log.
6. The method for displaying the logs of the distributed system in real time according to any one of claims 1 to 5, wherein the writing the encoded logs into the first buffer queue and the second buffer queue in sequence comprises:
writing the encoding log into a first cache queue;
if the queue length occupied by the coding logs in the first cache queue is smaller than the preset length, continuing to write the coding logs into the first cache queue;
and if the queue length occupied by the coding logs in the first cache queue is greater than or equal to the preset length, writing the coding logs into a second cache queue.
7. The method for real-time presentation of the logs of the distributed system according to any one of claims 1 to 5, wherein the real-time writing of the encoded logs in the first cache queue into a presentation page for log presentation by using the data link includes:
classifying the code logs in the first cache queue according to the unique codes to obtain display logs corresponding to the subsystems;
arranging the display logs in an ascending order according to the unique codes to obtain a log display sequence;
and writing the log display sequence into a preset display page for log display.
8. A distributed system log real-time display device is characterized in that the device comprises:
the queue creating module is used for creating a first buffer queue and a second buffer queue;
the log coding module is used for acquiring system logs of a plurality of background systems in the distributed system and respectively carrying out unique index coding on the system logs of each background system to obtain coded logs;
the log caching module is used for sequentially writing the encoded logs into the first caching queue and the second caching queue;
and the log display module is used for writing the coded logs in the first cache queue into a preset display page in real time for log display when the queue length occupied by the coded logs in the second cache queue is greater than a preset queue threshold value.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of real-time presentation of distributed system logs as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the distributed system log real-time presentation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110483811.1A CN113111137A (en) | 2021-04-30 | 2021-04-30 | Distributed system log real-time display method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110483811.1A CN113111137A (en) | 2021-04-30 | 2021-04-30 | Distributed system log real-time display method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113111137A true CN113111137A (en) | 2021-07-13 |
Family
ID=76720893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110483811.1A Pending CN113111137A (en) | 2021-04-30 | 2021-04-30 | Distributed system log real-time display method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113111137A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060224634A1 (en) * | 2005-03-31 | 2006-10-05 | Uwe Hahn | Multiple log queues in a database management system |
| CN101087210A (en) * | 2007-05-22 | 2007-12-12 | 网御神州科技（北京）有限公司 | High-performance Syslog processing and storage method |
| CN102521269A (en) * | 2011-11-22 | 2012-06-27 | 清华大学 | Index-based computer continuous data protection method |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
| CN108399186A (en) * | 2018-01-12 | 2018-08-14 | 联动优势科技有限公司 | A kind of collecting method and device |
| CN108829342A (en) * | 2018-05-09 | 2018-11-16 | 青岛海信宽带多媒体技术有限公司 | A kind of log storing method, system and storage device |
| CN110011845A (en) * | 2019-03-29 | 2019-07-12 | 新华三大数据技术有限公司 | Log collection method and system |
| CN111796983A (en) * | 2020-06-23 | 2020-10-20 | 中体彩科技发展有限公司 | System and method for monitoring abnormal transaction request of sportsbook |
| CN112104662A (en) * | 2020-09-27 | 2020-12-18 | 平安普惠企业管理有限公司 | Far-end data read-write method, device, equipment and computer readable storage medium |
-
2021
- 2021-04-30 CN CN202110483811.1A patent/CN113111137A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060224634A1 (en) * | 2005-03-31 | 2006-10-05 | Uwe Hahn | Multiple log queues in a database management system |
| CN101087210A (en) * | 2007-05-22 | 2007-12-12 | 网御神州科技（北京）有限公司 | High-performance Syslog processing and storage method |
| CN102521269A (en) * | 2011-11-22 | 2012-06-27 | 清华大学 | Index-based computer continuous data protection method |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
| CN108399186A (en) * | 2018-01-12 | 2018-08-14 | 联动优势科技有限公司 | A kind of collecting method and device |
| CN108829342A (en) * | 2018-05-09 | 2018-11-16 | 青岛海信宽带多媒体技术有限公司 | A kind of log storing method, system and storage device |
| CN110011845A (en) * | 2019-03-29 | 2019-07-12 | 新华三大数据技术有限公司 | Log collection method and system |
| CN111796983A (en) * | 2020-06-23 | 2020-10-20 | 中体彩科技发展有限公司 | System and method for monitoring abnormal transaction request of sportsbook |
| CN112104662A (en) * | 2020-09-27 | 2020-12-18 | 平安普惠企业管理有限公司 | Far-end data read-write method, device, equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112052370A (en) | Message generation method and device, electronic equipment and computer readable storage medium | |
| CN112347042A (en) | File uploading method and device, electronic equipment and storage medium | |
| CN113239106B (en) | Excel file export method and device, electronic equipment and storage medium | |
| CN112115145A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN112653760A (en) | Cross-server file transmission method and device, electronic equipment and storage medium | |
| CN112528307A (en) | Service request checking method and device, electronic equipment and storage medium | |
| CN114185895A (en) | Data import and export method and device, electronic equipment and storage medium | |
| CN112084486A (en) | User information verification method and device, electronic equipment and storage medium | |
| CN112699142A (en) | Cold and hot data processing method and device, electronic equipment and storage medium | |
| CN112580079A (en) | Authority configuration method and device, electronic equipment and readable storage medium | |
| CN112464619B (en) | Big data processing method, device and equipment and computer readable storage medium | |
| CN114844844A (en) | Delay message processing method, device, equipment and storage medium | |
| CN114398346A (en) | Data migration method, device, equipment and storage medium | |
| CN113438304A (en) | Data query method, device, server and medium based on database cluster | |
| CN112685384A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN116842012A (en) | Method, device, equipment and storage medium for storing Redis cluster in fragments | |
| CN112256472A (en) | Distributed data calling method and device, electronic equipment and storage medium | |
| CN113111137A (en) | Distributed system log real-time display method, device, equipment and storage medium | |
| CN114840631A (en) | Spatial text query method and device, electronic equipment and storage medium | |
| CN115033605A (en) | Data query method and device, electronic equipment and storage medium | |
| CN115174555A (en) | File transmission method and device, electronic equipment and storage medium | |
| CN115145870A (en) | Method and device for positioning reason of failed task, electronic equipment and storage medium | |
| CN113918517A (en) | Multi-type file centralized management method, device, equipment and storage medium | |
| CN113364848A (en) | File caching method and device, electronic equipment and storage medium | |
| CN114417998A (en) | Data feature mapping method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40051231 Country of ref document: HK |