CN113760657A - Log monitoring method, device, equipment and storage medium - Google Patents
Log monitoring method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113760657A CN113760657A CN202111019474.7A CN202111019474A CN113760657A CN 113760657 A CN113760657 A CN 113760657A CN 202111019474 A CN202111019474 A CN 202111019474A CN 113760657 A CN113760657 A CN 113760657A
- Authority
- CN
- China
- Prior art keywords
- log
- monitoring
- training
- algorithm training
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 209
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 claims abstract description 243
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 165
- 238000004590 computer program Methods 0.000 claims description 13
- 230000002159 abnormal effect Effects 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 4
- 238000012806 monitoring device Methods 0.000 abstract description 9
- 238000012360 testing method Methods 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3055—Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a log monitoring method, a log monitoring device, log monitoring equipment and a log monitoring storage medium. The method comprises the steps of acquiring a real-time training log generated by an algorithm training platform within a set log monitoring time range; generating monitoring warning information of an algorithm training state according to the real-time training log; and feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information. The technical scheme of the embodiment of the invention can save the time and labor cost of algorithm training and effectively improve the testing efficiency of algorithm training.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a log monitoring method, a log monitoring device, log monitoring equipment and a log monitoring storage medium.
Background
The daily algorithm training task can be carried out for several hours or even several days, after the algorithm training task is started, the log of the training start cannot be visually seen on the front end page, the training result can only be checked after the training is finished, and the training is restarted if the training is failed or invalid.
In order to prevent the time waste of failed training or invalid training caused by the fact that the training logs cannot be checked through the front-end page, a tester usually refers to the logs in the middle and early stages of algorithm training manually, judges whether the follow-up of the algorithm training task can be successfully trained and completed through self experience, if the early-stage logs report errors, the follow-up training is invalid with high probability, and if the early-stage logs do not report errors or little error-reporting information, the follow-up training can be normally carried out.
In the process of implementing the invention, the inventor finds that the prior art has the following defects: the training mode, in which the training result is obtained only after the training of the algorithm training task is completed, not only wastes computing resources, but also consumes manpower and time. The method of judging the success probability of the subsequent algorithm training by manually consulting the early-stage log of the training consumes manpower and time, and the above method is not beneficial to improving the efficiency of the algorithm training.
Disclosure of Invention
The embodiment of the invention provides a log monitoring method, a log monitoring device, log monitoring equipment and a log monitoring storage medium, which can save the time and labor cost of algorithm training, thereby effectively improving the testing efficiency of the algorithm training.
In a first aspect, an embodiment of the present invention provides a log monitoring method, including:
acquiring a real-time training log generated by an algorithm training platform within a set log monitoring time range;
generating monitoring warning information of an algorithm training state according to the real-time training log;
and feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
In a second aspect, an embodiment of the present invention further provides a log monitoring apparatus, including:
the log acquisition module is used for acquiring a real-time training log generated by the algorithm training platform within a set log monitoring time range;
the information generation module is used for generating monitoring warning information of the algorithm training state according to the real-time training log;
and the state determining module is used for feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
In a third aspect, an embodiment of the present invention further provides a computer device, including:
one or more processors;
storage means for storing one or more computer programs;
the log monitoring method provided by any embodiment of the invention is implemented when the one or more computer programs are executed by the one or more processors, so that the one or more processors execute the computer programs.
In a fourth aspect, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the log monitoring method provided in any embodiment of the present invention.
The embodiment of the invention generates the monitoring warning information of the algorithm training state according to the real-time training log by acquiring the real-time training log generated by the algorithm training platform within the set log monitoring time range, so that the monitoring warning information is fed back to the user, the user can determine the algorithm training state according to the monitoring warning information, the problems of low algorithm training efficiency and the like in the existing algorithm training that whether the algorithm training task can be successfully trained and completed is judged by manually checking the log are solved, the invalid training times of the algorithm can be effectively reduced, the calculation resources of the algorithm training and the time and labor cost are saved, and the testing efficiency of the algorithm training is effectively improved.
Drawings
Fig. 1 is a schematic diagram of a log monitoring system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an operating principle of a log monitoring system according to an embodiment of the present invention;
fig. 3 is a flowchart of a log monitoring method according to a second embodiment of the present invention;
fig. 4 is a schematic diagram illustrating an effect of the output information of the warning module according to the second embodiment of the present invention;
FIG. 5 is a schematic diagram of a log monitoring apparatus according to a third embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention.
It should be further noted that, for the convenience of description, only some but not all of the relevant aspects of the present invention are shown in the drawings. Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The terms "first" and "second," and the like in the description and claims of embodiments of the invention and in the drawings, are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not set forth for a listed step or element but may include steps or elements not listed.
Example one
Fig. 1 is a schematic diagram of a log monitoring system according to an embodiment of the present invention, where the structure of the log monitoring system includes: a log storage module 110, a monitoring module 120 and an alert module 130.
The log storage module 110 may be specifically configured to store real-time log information of an algorithm training platform training algorithm task.
The monitoring module 120 may be specifically configured to interface with the log storage module 110, collect real-time log information of training, count the number of error messages in the log by itself, and send the statistical result to the warning module 130.
For example, after the algorithm is successfully trained on the machine learning algorithm platform, a task is started in a docker container corresponding to the background server, and then the monitoring module 120 reads a real-time training log in the container, and determines the state of the algorithm training by capturing error-reporting keywords and/or abnormal events in the real-time training log. If multiple errors, fail or user-defined error-reporting keywords occur, the real-time training log and the amount of error-reporting information are sent to the warning module 130.
The warning module 130, which may be a front-end page displayed to the user, may be interconnected with the monitoring module 120, receive the number of error reporting messages processed and counted by the monitoring module 120, and display the number of error reporting messages to the user to give a prompt. Before the log monitoring system monitors the algorithm training task in the algorithm training platform, the warning module 130 may also support the user to define the time length required to be monitored and trained and the defined error-reporting keyword required to be monitored, and notify the monitoring module 120, so as to implement the user-defined configuration of the related monitoring parameters of the log monitoring system.
Fig. 2 is a schematic diagram of an operating principle of a log monitoring system according to an embodiment of the present invention. In a specific example, as shown in fig. 2, in the process of performing an algorithm or model training task, the monitoring module obtains real-time training log information in the log storage module of the algorithm training platform, monitors error information of the algorithm training task within a set monitoring log time, counts the number of the error information, transmits the error information to the warning module, displays the error information to the user, and gives a prompt to prompt the user about the subsequent success rate of the algorithm training task. Before monitoring the training task, a user can customize the set monitoring log time of the log monitoring system and the error-reporting keywords to be monitored through the warning module, and inform the monitoring module of the set monitoring log time and the error-reporting keywords to be monitored.
The working principle of the log monitoring system in the embodiment of the invention is as follows: in the process of performing the algorithm or model training task, the monitoring module 120 obtains real-time log information in the training platform log storage module 110, monitors log error reporting information of the algorithm training task within the monitoring duration, counts the number of the error reporting information, transmits the error reporting information to the warning module 120, displays the error reporting information to the user, and gives a prompt to prompt the user about the subsequent success rate of the algorithm training task.
According to the technical scheme, the log monitoring system is formed by the monitoring module and the warning module, the real-time training log generated by the algorithm training platform within the set log monitoring time range is obtained through the log monitoring system, the monitoring warning information of the algorithm training state is generated according to the real-time training log, the monitoring warning information is fed back to a user, the algorithm training state is determined by the user according to the monitoring warning information, the problems that the algorithm training efficiency is low and the like when the existing algorithm training judges whether the algorithm training task can be successfully trained through manually checking the log are solved, the invalid training times of the algorithm can be effectively reduced, the calculation resources and the time and labor cost of the algorithm training are saved, and the testing efficiency of the algorithm training is effectively improved.
Example two
Fig. 3 is a flowchart of a log monitoring method according to a second embodiment of the present invention, where this embodiment is applicable to prompt a user of a subsequent success rate of an algorithm training task of this time according to error reporting information of a log at an initial stage of the algorithm training task, and the method may be executed by a log monitoring device, where the device may be implemented in a software and/or hardware manner, and may be generally integrated in a server, as shown in fig. 3, where the method includes the following operations:
s310, acquiring a real-time training log generated by the algorithm training platform within a set log monitoring time range.
The algorithm training platform can be an operating environment of an algorithm training task and can be used for training algorithms such as machine learning or deep learning. The set log monitoring time can be preset, and the time for monitoring the real-time training log of the algorithm training platform can be a default value or can be customized by a user. For example, the set log monitoring time may be the first 30 minutes of the algorithm training task by default, or may be customized by the user as the first 12 hours of the algorithm training task, which is not limited in this embodiment. It should be noted that if the log monitoring time is not set, the log monitoring time may be set by default to be the real-time training log within 30 minutes after the algorithm training task starts. The real-time training log can be recorded information synchronously generated by the algorithm in the training process, can reflect the problems of the algorithm training task in the training process, and can be used for judging whether the algorithm training task can be successfully trained and completed subsequently according to the real-time training log.
In the embodiment of the invention, the log monitoring system needs to obtain the real-time training log of the algorithm training task in the algorithm training platform in the set monitoring time. That is, the log monitoring system can complete the real-time monitoring of the algorithm training platform only after acquiring the real-time training log. It should be noted that training durations of different algorithm training tasks are different, and a user needs to determine an optimal set log monitoring time according to different scenes and different algorithm training tasks, that is, the user can set the log monitoring time in a user-defined manner.
In an optional embodiment of the present invention, before obtaining the real-time training log generated by the algorithm training platform within the set log monitoring time range, the method may include: acquiring a monitoring time setting instruction input by the user; and determining the set log monitoring time according to the monitoring time setting instruction.
The monitoring time setting instruction may be a time setting instruction and a command of the log monitoring system by the user, and exemplarily, the monitoring time setting instruction may be a specific time value set by the user at the self-defined log monitoring time of the log monitoring system.
Before the log monitoring system obtains the real-time training log, a user can input a monitoring time setting instruction to the log monitoring system, and the log monitoring system can determine the log monitoring time set by the user for the real-time training log according to the monitoring time setting instruction of the user.
And S320, generating monitoring warning information of the algorithm training state according to the real-time training log.
The algorithm training state may be a current algorithm training state. The monitoring warning information can be information for prompting the user of the early-stage state of the algorithm training, and can comprise the degree of the number of error-reporting information, so as to warn the user of the success rate of the algorithm training. The error reporting information can indicate a prompt given when the algorithm training task has an error. The number of error reporting messages is also the number of prompt messages for errors in the algorithm training task.
In the embodiment of the invention, the log monitoring system can monitor the real-time training log and count the error reporting information recorded in the real-time training log so as to generate the monitoring warning information according to the statistical result.
In an optional embodiment of the present invention, the generating of the monitoring warning information of the training status of the algorithm according to the real-time training log may include: determining error-reporting keywords and/or abnormal error-reporting events according to the real-time training log; counting the error reporting information quantity of the error reporting keywords and/or the abnormal error reporting events; and generating monitoring warning information of the algorithm training state according to the error reporting information quantity.
The error-reporting keywords can be identification characters with warning significance, and can include default error-reporting keywords and/or custom error-reporting keywords. Illustratively, the default error-reporting keyword may be an error field, an Exception field, or the like, and the user-defined error-reporting keyword may be flexibly determined by the user according to the algorithm training task, which is not limited in this embodiment. The exception error reporting event can be an exception event occurring in the operation of the log monitoring system and can be caused by hardware or software design problems. The error information amount can be obtained by adding the error keywords and/or the number of abnormal error events.
Correspondingly, the log monitoring system acquires error-reporting keywords and/or abnormal error-reporting events in the real-time training log, counts the number of the error-reporting keywords and displays the statistical result to a user for warning.
In an optional embodiment of the present invention, before obtaining the real-time training log generated by the algorithm training platform within the set log monitoring time range, the method may further include: acquiring a user-defined keyword setting instruction input by the user; and determining a custom error-reporting keyword according to the custom keyword setting instruction.
The user-defined keyword setting instruction can be specific words of user-defined error-reporting keywords of the log monitoring system.
Before the log monitoring system acquires the real-time training log, a user sends a user-defined keyword setting instruction to the log monitoring system, and the log monitoring system receives and executes the instruction of the user. The method has the advantages that the error-reporting keyword ranges of different algorithm training tasks can be flexibly determined, and the accuracy of training and monitoring different scenes of different users is improved.
In an optional embodiment of the present invention, the generating the monitoring warning information of the algorithm training status according to the number of error messages may include: generating first monitoring warning information under the condition that the number of the error reporting information is determined to be less than or equal to a first preset number; generating second monitoring warning information under the condition that the number of the error reporting information is determined to be larger than the first preset number and smaller than a second preset number; and generating third monitoring warning information under the condition that the number of the error reporting information is determined to be greater than or equal to the second preset number.
The first preset number may be an upper limit value of a first level of the number of error reporting messages, and is used for performing degree differentiation on the number of error reporting messages. The first monitoring warning information can be information of a first level generated by the log monitoring system when the number of error reporting information does not exceed a first preset number, and is displayed by the warning module. The second predetermined number may be an upper limit value of a second level of the number of error messages, and it is understood that the second predetermined number is greater than the first predetermined number. The second monitoring warning information may be a second level of information generated by the log monitoring system when the number of error reporting information exceeds the first preset number and does not exceed the second preset number, and the second monitoring warning information is stronger than the first monitoring warning information. The third monitoring warning information may be information of a third level generated by the log monitoring system when the number of the error-reporting information exceeds a second preset number, and the third monitoring warning information is stronger than the second monitoring warning information.
In the embodiment of the invention, the log monitoring system compares the counted error-reporting information quantity with a first preset quantity and a second preset quantity, and generates corresponding monitoring warning information according to the comparison result.
In an alternative embodiment of the invention, the monitoring alert information includes alert level information and/or algorithm training recommendation information.
The warning degree information can be a degree prompt of the number of error reporting information, and can include slight, general and serious. The algorithm training suggestion information can be error degree prompts about current algorithm training and suggestion information about subsequent algorithm training which are displayed to a user according to the warning degree information warning module. The monitoring warning information can be the comprehensive embodiment of the warning degree information and the algorithm training suggestion.
In a specific example, fig. 4 is a schematic view illustrating an effect of the output information of the warning module provided in the second embodiment of the present invention, as shown in fig. 4, for example, if the number of error messages is 1 to 3, a first monitoring warning message of "slight (1-3), and slight degree failure risk exists in the algorithm training task of this time" is prompted; if the number of the error-reporting messages is 4-10, prompting a second monitoring warning message of 'general (4-10) and general degree failure risk of the algorithm training task at this time'; and if the number of the error-reporting messages is more than 10, prompting a third monitoring warning message of ' serious (more than 10) ', the algorithm training task of this time has the risk of serious failure, and advising you to initiate training again after checking the task '.
S330, feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
Correspondingly, the log monitoring system returns the acquired monitoring warning information to the user, and the user determines whether the algorithm training task continues to be trained according to the warning degree information prompted by the monitoring warning information and the algorithm training suggestion.
In an optional embodiment of the present invention, after the feeding back the monitoring alert information to the user, the method may include: receiving an interrupt algorithm training instruction sent by the user; and interrupting the algorithm task currently trained by the algorithm training platform in response to the interruption algorithm training instruction.
The interrupt algorithm training instruction may be an instruction sent by a user to the log monitoring system to instruct the algorithm training task in training to stop training.
Optionally, the user may determine whether the ongoing algorithm training task continues according to the fed-back monitoring warning information, and if the user determines that the algorithm training task needs to stop training, the user may send an interrupt algorithm training instruction to the log monitoring system, and the log monitoring system receives an instruction of the user to perform an interrupt operation on the current training task of the algorithm training platform. Illustratively, if the log monitoring system prompts that "serious (more than 10) is existed, the algorithm training task of this time has a risk of serious failure, it is recommended that training is initiated again after the task is checked, the user can determine that the algorithm training task needs to be stopped training, an algorithm training interruption instruction is sent to the log monitoring system, and the log monitoring system executes the user instruction to interrupt the algorithm training task in training. Alternatively, if the user determines that the algorithm training task needs to be stopped, the user may also directly control the algorithm training platform to interrupt the current training task.
According to the technical scheme of the embodiment, the real-time training log generated by the algorithm training platform within the set log monitoring time range is obtained, the monitoring warning information of the algorithm training state is generated according to the real-time training log, the monitoring warning information is fed back to the user, the user is enabled to determine the algorithm training state according to the monitoring warning information, the problems that the algorithm training efficiency is low and the like when the algorithm training is judged whether the algorithm training task can be successfully trained and completed through manually checking the log in the existing algorithm training are solved, the invalid training times of the algorithm can be effectively reduced, the calculation resources and the time and labor cost of the algorithm training are saved, and the testing efficiency of the algorithm training is effectively improved.
EXAMPLE III
Fig. 5 is a schematic diagram of a log monitoring apparatus according to a third embodiment of the present invention, and as shown in fig. 5, the apparatus includes: a log obtaining module 510, an information generating module 520, and a status determining module 530, wherein:
a log obtaining module 510, configured to obtain a real-time training log generated by the algorithm training platform within a set log monitoring time range;
an information generating module 520, configured to generate monitoring warning information of an algorithm training state according to the real-time training log;
a state determining module 530, configured to feed the monitoring warning information back to the user, so that the user determines the algorithm training state according to the monitoring warning information.
According to the technical scheme of the embodiment, the real-time training log generated by the algorithm training platform within the set log monitoring time range is obtained, the monitoring warning information of the algorithm training state is generated according to the real-time training log, the monitoring warning information is fed back to the user, the user is enabled to determine the algorithm training state according to the monitoring warning information, the problems that the algorithm training efficiency is low and the like when the algorithm training is judged whether the algorithm training task can be successfully trained and completed through manually checking the log in the existing algorithm training are solved, the invalid training times of the algorithm can be effectively reduced, the calculation resources and the time and labor cost of the algorithm training are saved, and the testing efficiency of the algorithm training is effectively improved.
In the above apparatus, optionally, the apparatus further includes: a monitoring time determining module used for determining the real-time training log generated by the acquisition algorithm training platform within the set log monitoring time range,
acquiring a monitoring time setting instruction input by the user;
and determining the set log monitoring time according to the monitoring time setting instruction.
In the above apparatus, optionally, the information generating module 520 further includes:
the error reporting determining unit is used for determining error reporting keywords and/or abnormal error reporting events according to the real-time training log;
the error reporting statistic unit is used for counting the number of error reporting information of the error reporting keywords and/or the abnormal error reporting events;
and the warning information generating unit is used for generating monitoring warning information of the algorithm training state according to the error reporting information quantity.
In the above apparatus, optionally, the error-reporting keyword includes a default error-reporting keyword and/or a custom error-reporting keyword;
the warning information generating unit further comprises:
the first warning information generating unit is used for generating first monitoring warning information under the condition that the number of the error reporting information is determined to be less than or equal to a first preset number;
the second warning information generating unit is used for generating second monitoring warning information under the condition that the number of the error reporting information is determined to be larger than the first preset number and smaller than a second preset number;
and the third warning information generating unit is used for generating third monitoring warning information under the condition that the number of the error reporting information is determined to be greater than or equal to the second preset number.
In the above apparatus, optionally, the monitoring alert information includes alert degree information and/or algorithm training suggestion information.
In the above apparatus, optionally, the apparatus further includes: a self-defined error-reporting keyword determining module used for generating a real-time training log in a set log monitoring time range before the real-time training log generated by the acquisition algorithm training platform,
acquiring a user-defined keyword setting instruction input by the user;
and determining a custom error-reporting keyword according to the custom keyword setting instruction.
In the above apparatus, optionally, the apparatus further includes: an interruption algorithm training instruction response module for feeding back the monitoring warning information to the user,
receiving an interrupt algorithm training instruction sent by the user;
and interrupting the algorithm task currently trained by the algorithm training platform in response to the interruption algorithm training instruction.
The log monitoring device can execute the log monitoring method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to a log monitoring method provided in any embodiment of the present invention.
Since the log monitoring device described above is a device capable of executing the log monitoring method in the embodiment of the present invention, based on the log monitoring method described in the embodiment of the present invention, a person skilled in the art can understand the specific implementation manner of the log monitoring device in the embodiment and various variations thereof, and therefore, how the log monitoring device implements the log monitoring method in the embodiment of the present invention is not described in detail herein. As long as those skilled in the art implement the apparatus used in the log monitoring method in the embodiments of the present invention, the apparatus is within the scope of the present application.
Example four
Fig. 6 is a schematic structural diagram of a computer apparatus according to a fourth embodiment of the present invention, as shown in fig. 6, the apparatus includes a processor 60, a storage device 61, an input device 62, and an output device 63; the number of processors 60 in the device may be one or more, and one processor 60 is taken as an example in fig. 6; the processor 60, the storage means 61, the input means 62 and the output means 63 in the device may be connected by a bus or other means, as exemplified by a bus connection in fig. 6.
The storage device 61, as a computer-readable storage medium, can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the log monitoring method in the embodiment of the present invention (for example, the log obtaining module 510, the information generating module 520, and the status determining module 530 in the log monitoring device). The processor 60 executes various functional applications and data processing of the device by executing software programs, instructions and modules stored in the storage device 61, so as to implement the log monitoring method, which includes:
acquiring a real-time training log generated by an algorithm training platform within a set log monitoring time range;
generating monitoring warning information of an algorithm training state according to the real-time training log;
and feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
The storage device 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the storage device 61 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the storage 61 may further include memory located remotely from the processor 60, which may be connected to the device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 62 may be used to receive entered numeric or character information and to generate key signal inputs relating to user settings and function controls of the apparatus. The output device 63 may include a display device such as a display screen.
EXAMPLE five
An embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform a log monitoring method, and the method includes:
acquiring a real-time training log generated by an algorithm training platform within a set log monitoring time range;
generating monitoring warning information of an algorithm training state according to the real-time training log;
and feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
Of course, the computer program executed by the computer storage medium provided by the embodiment of the present invention is not limited to the method operations described above, and may also execute related operations in the log monitoring method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the log monitoring apparatus, each included unit and module are only divided according to functional logic, but are not limited to the above division as long as the corresponding function can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A log monitoring method, comprising:
acquiring a real-time training log generated by an algorithm training platform within a set log monitoring time range;
generating monitoring warning information of an algorithm training state according to the real-time training log;
and feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
2. The method of claim 1, wherein before obtaining the real-time training logs generated by the algorithm training platform within the set log monitoring time range, further comprising:
acquiring a monitoring time setting instruction input by the user;
and determining the set log monitoring time according to the monitoring time setting instruction.
3. The method of claim 1, wherein generating the monitoring alert information of the algorithm training status according to the real-time training log comprises:
determining error-reporting keywords and/or abnormal error-reporting events according to the real-time training log;
counting the error reporting information quantity of the error reporting keywords and/or the abnormal error reporting events;
and generating monitoring warning information of the algorithm training state according to the error reporting information quantity.
4. The method of claim 3, wherein the error-reporting key comprises a default error-reporting key and/or a custom error-reporting key;
the generating of the monitoring warning information of the algorithm training state according to the error reporting information quantity comprises the following steps:
generating first monitoring warning information under the condition that the number of the error reporting information is determined to be less than or equal to a first preset number;
generating second monitoring warning information under the condition that the number of the error reporting information is determined to be larger than the first preset number and smaller than a second preset number;
and generating third monitoring warning information under the condition that the number of the error reporting information is determined to be greater than or equal to the second preset number.
5. The method of claim 4, wherein the monitoring alert information comprises alert level information and/or algorithm training recommendation information.
6. The method according to any one of claims 3-5, wherein before obtaining the real-time training log generated by the algorithm training platform within the set log monitoring time range, the method further comprises:
acquiring a user-defined keyword setting instruction input by the user;
and determining a custom error-reporting keyword according to the custom keyword setting instruction.
7. The method of claim 1, further comprising, after said feeding back the monitoring alert information to the user:
receiving an interrupt algorithm training instruction sent by the user;
and interrupting the algorithm task currently trained by the algorithm training platform in response to the interruption algorithm training instruction.
8. A log monitoring apparatus, comprising:
the log acquisition module is used for acquiring a real-time training log generated by the algorithm training platform within a set log monitoring time range;
the information generation module is used for generating monitoring warning information of the algorithm training state according to the real-time training log;
and the state determining module is used for feeding the monitoring warning information back to a user so that the user can determine the algorithm training state according to the monitoring warning information.
9. A computer device, characterized in that the computer device comprises:
one or more processors;
storage means for storing one or more computer programs;
the log monitoring method as claimed in any one of claims 1-7, when executed by the one or more computer programs to cause the one or more processors to execute the computer programs.
10. A computer storage medium having a computer program stored thereon, the computer program, when being executed by a processor, implementing a log monitoring method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111019474.7A CN113760657A (en) | 2021-09-01 | 2021-09-01 | Log monitoring method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111019474.7A CN113760657A (en) | 2021-09-01 | 2021-09-01 | Log monitoring method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113760657A true CN113760657A (en) | 2021-12-07 |
Family
ID=78792456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111019474.7A Pending CN113760657A (en) | 2021-09-01 | 2021-09-01 | Log monitoring method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113760657A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117785177A (en) * | 2023-12-26 | 2024-03-29 | 北京汉勃科技有限公司 | Visual large model fine tuning method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175679A (en) * | 2019-05-29 | 2019-08-27 | 深圳前海微众银行股份有限公司 | A kind of method and device of monitoring model training |
| US20200074984A1 (en) * | 2018-08-31 | 2020-03-05 | International Business Machines Corporation | Intent authoring using weak supervision and co-training for automated response systems |
| CN111143207A (en) * | 2019-12-19 | 2020-05-12 | 北京智能工场科技有限公司 | Method for checking model training notice and training log at mobile terminal |
-
2021
- 2021-09-01 CN CN202111019474.7A patent/CN113760657A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200074984A1 (en) * | 2018-08-31 | 2020-03-05 | International Business Machines Corporation | Intent authoring using weak supervision and co-training for automated response systems |
| CN110175679A (en) * | 2019-05-29 | 2019-08-27 | 深圳前海微众银行股份有限公司 | A kind of method and device of monitoring model training |
| CN111143207A (en) * | 2019-12-19 | 2020-05-12 | 北京智能工场科技有限公司 | Method for checking model training notice and training log at mobile terminal |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117785177A (en) * | 2023-12-26 | 2024-03-29 | 北京汉勃科技有限公司 | Visual large model fine tuning method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109726072B (en) | WebLogic server monitoring and alarming method, device and system and computer storage medium | |
| CN110224858B (en) | Log-based alarm method and related device | |
| CN111882833B (en) | Equipment fault early warning method, device, equipment and medium based on outlier parameters | |
| JP4502414B2 (en) | Production management information output device and production management information output method | |
| CN113760657A (en) | Log monitoring method, device, equipment and storage medium | |
| CN111130867B (en) | Intelligent household equipment alarm method and device based on Internet of things | |
| CN115718450A (en) | Equipment wire-stopping monitoring method and device, electronic equipment and system | |
| CN110674149A (en) | Service data processing method and device, computer equipment and storage medium | |
| CN111949421B (en) | SDK calling method, device, electronic equipment and computer readable storage medium | |
| CN111124818B (en) | Monitoring method, device and equipment for Expander | |
| CN114610560B (en) | System abnormality monitoring method, device and storage medium | |
| CN107145402B (en) | Method for detecting software downtime and electronic equipment | |
| CN113220379B (en) | Task processing method and device, electronic equipment and readable storage medium | |
| CN112804104A (en) | Early warning method, device, equipment and medium | |
| CN113507721A (en) | Network state monitoring method, device, system and storage medium | |
| CN116416764A (en) | Alarm threshold generation method and device, electronic equipment and storage medium | |
| CN113592108A (en) | Charging fault management method of charging pile and related equipment | |
| CN112069027A (en) | Interface data processing method and device, electronic equipment and storage medium | |
| CN111679943A (en) | Server test system | |
| CN111930594A (en) | Method and device for monitoring down of target machine and readable storage equipment | |
| CN112232765A (en) | Personnel reminding method and device, electronic equipment and storage medium | |
| CN113778836B (en) | Cloud native application health monitoring method, device, equipment and readable storage medium | |
| CN115792606B (en) | Water pump motor fault detection method, device, equipment and storage medium | |
| CN116260963B (en) | Emergency handling method and device for camera fault | |
| CN116915648A (en) | Interface monitoring method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 210000 building C4, Hongfeng Science Park, Nanjing Economic and Technological Development Zone, Nanjing City, Jiangsu Province Applicant after: NANQI XIANCE (NANJING) TECHNOLOGY Co.,Ltd. Address before: 210000 floor 17, building 32, headquarters base, Jiangning District, Nanjing, Jiangsu Province Applicant before: NANQI XIANCE (NANJING) TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information |