CN118312383A - Device state determining method and device, nonvolatile storage medium and electronic device - Google Patents
Device state determining method and device, nonvolatile storage medium and electronic device Download PDFInfo
- Publication number
- CN118312383A CN118312383A CN202410418453.XA CN202410418453A CN118312383A CN 118312383 A CN118312383 A CN 118312383A CN 202410418453 A CN202410418453 A CN 202410418453A CN 118312383 A CN118312383 A CN 118312383A
- Authority
- CN
- China
- Prior art keywords
- log
- equipment
- semantic
- word
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 230000002159 abnormal effect Effects 0.000 claims abstract description 97
- 239000013598 vector Substances 0.000 claims description 141
- 230000011218 segmentation Effects 0.000 claims description 51
- 238000012545 processing Methods 0.000 claims description 46
- 238000012549 training Methods 0.000 claims description 37
- 230000015654 memory Effects 0.000 claims description 24
- 238000007781 pre-processing Methods 0.000 claims description 8
- 230000001965 increasing effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 12
- 238000001514 detection method Methods 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000012423 maintenance Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000005856 abnormality Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- ABEXEQSGABRUHS-UHFFFAOYSA-N 16-methylheptadecyl 16-methylheptadecanoate Chemical compound CC(C)CCCCCCCCCCCCCCCOC(=O)CCCCCCCCCCCCCCC(C)C ABEXEQSGABRUHS-UHFFFAOYSA-N 0.000 description 1
- 206010010254 Concussion Diseases 0.000 description 1
- 241000764238 Isis Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000009514 concussion Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000005417 image-selected in vivo spectroscopy Methods 0.000 description 1
- 238000012739 integrated shape imaging system Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The application discloses a device state determining method and device, a nonvolatile storage medium and electronic equipment. Wherein the method comprises the following steps: determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log theme word stock; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; and carrying out content identification on the device log through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each device in a preset time period according to the identification result. The method and the device solve the technical problem that the device state cannot be accurately determined in a scene with diversified log formats due to high requirements on the log formats when the device state is judged according to the log in the related technology.
Description
Technical Field
The present application relates to the field of operation and maintenance, and in particular, to a device state determining method and apparatus, a nonvolatile storage medium, and an electronic device.
Background
In the related art, when determining the working state of a device in a network through a log, the log is generally required to have a fixed format, and complex and accurate log judgment rules are adopted to determine the content of the log so as to determine the state of the device. However, the devices in the network may be provided by different manufacturers, so the log formats generated by different devices are not uniform, and the adaptive judgment rules are not the same. This results in that the related art cannot well realize the identification of the device state according to the log in the scene of log format diversification. And the related art can only judge the current state of the equipment or the historical state of the equipment according to the log, and can not predict the future state of the equipment.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the application provides a device state determining method and device, a nonvolatile storage medium and electronic equipment, which at least solve the technical problem that the device state cannot be accurately determined in a scene with diversified log formats due to high requirements on the log formats when the device state is judged according to the log in the related technology.
According to an aspect of an embodiment of the present application, there is provided a device state determining method, including: acquiring device logs of all devices in a network system; determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises scores of keywords and keywords in the log text, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; and carrying out content recognition on the equipment logs through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each equipment in a preset time period according to a recognition result, wherein a training dataset of the multi-scale log semantic understanding device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
Optionally, determining the abstract text of the device log according to the syntactic dependency relationship and semantic information of the log text in the device log and the preset log topic word stock includes: after preprocessing the device log, determining a directed acyclic graph according to the syntactic dependency relationship and semantic information of the log text in the preprocessed device log, wherein nodes in the directed acyclic graph represent log data words in the preprocessed log text, edges between the nodes represent association relationships between the log data words corresponding to the nodes, and the association relationships comprise at least one of the following: syntactic dependencies, semantic relationships; determining keywords in the preprocessed log text and scores of the keywords according to a preset log theme word stock and a directed acyclic graph; and generating abstract text according to the keywords and the scores.
Optionally, the edges in the directed acyclic graph are further used for indicating edge weight information, wherein the edge weight information is used for indicating the association degree between two log data words corresponding to the edges.
Optionally, determining the keywords in the preprocessed log text according to the preset log topic word stock and the directed acyclic graph, and scoring the keywords includes: determining target log data words contained in a preset log theme word stock from all log data words indicated by the directed acyclic graph; determining target log data words as keywords, and increasing the weight of the keywords, wherein the weight is used for reflecting the importance degree of the keywords; and determining the score according to the weight of the keyword.
Optionally, performing content recognition on the device log through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each device in a preset time period according to the recognition result includes: processing the device log through the word segmentation model to obtain a word vector corresponding to the device log and a semantic attention weight value of the word vector, wherein the semantic attention weight value is used for reflecting the importance degree of the word vector; determining a global semantic understanding feature vector of the device log according to the word vector and the semantic attention weight value of the word vector; and processing the global semantic understanding feature vector through a preset multi-scale log semantic understanding device to obtain a classification result, wherein the classification result is a working state prediction result and is used for indicating the working state of equipment corresponding to the equipment log in a preset time period.
Optionally, the word vector comprises a word granularity semantic connection feature vector; processing the device log through the word segmentation model to obtain a word vector corresponding to the device log, wherein the semantic attention weight value of the word vector comprises the following steps: after word segmentation is carried out on the device logs through a word segmentation model, the device logs subjected to word segmentation are processed through a forward long-short-term memory network to obtain first-scale semantic coding feature vectors, and the device logs subjected to word segmentation are processed through a reverse long-short-term memory network to obtain second-scale semantic coding feature vectors; fusing the first scale word semantic coding feature vector and the second scale semantic coding feature vector to obtain a multi-scale semantic coding feature vector, wherein the multi-scale semantic coding feature vector comprises word granularity semantic connection feature vectors; and determining the weight value of each word granularity semantic connection feature vector relative to all word granularity semantic connection feature vectors as a semantic attention weight value.
Optionally, determining the global semantic understanding feature vector of the device log according to the word vector and the semantic attention weight value of the word vector comprises: determining a first sequence and a second sequence, wherein the first sequence comprises semantic attention weight values of semantic connection feature vectors with each word granularity, and the second sequence comprises semantic connection feature vectors with each word granularity; and fusing semantic connection feature vectors of each word granularity in the second sequence according to the first sequence to obtain a global semantic understanding feature vector.
According to another aspect of the embodiment of the present application, there is also provided an apparatus for determining a device status, including: the first processing module is used for acquiring device logs of all devices in the network system; the second processing module is used for determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises keywords in the log text and scores of the keywords, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; the third processing module is used for determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; the fourth processing module is used for carrying out content recognition on the equipment logs through a preset multi-scale log semantic comprehension device, and determining a working state prediction result of each equipment in a preset time period according to the recognition result, wherein a training data set of the multi-scale log semantic comprehension device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
According to another aspect of the embodiment of the present application, there is also provided an apparatus for determining a device status, including: the first processing module is used for acquiring device logs of all devices in the network system; the second processing module is used for determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises keywords in the log text and scores of the keywords, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; the third processing module is used for determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; the fourth processing module is used for carrying out content recognition on the equipment logs through a preset multi-scale log semantic comprehension device, and determining a working state prediction result of each equipment in a preset time period according to the recognition result, wherein a training data set of the multi-scale log semantic comprehension device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
According to another aspect of the embodiments of the present application, there is provided a nonvolatile storage medium in which a program is stored, wherein the device in which the nonvolatile storage medium is controlled to execute a device state determining method when the program runs.
According to another aspect of an embodiment of the present application, there is provided an electronic apparatus including: the device comprises a memory and a processor for running a program stored in the memory, wherein the program executes a device state determining method when running.
According to another aspect of embodiments of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a device state determination method.
In the embodiment of the application, the method comprises the steps of acquiring the equipment logs of all the equipment in the network system; determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises scores of keywords and keywords in the log text, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; the method comprises the steps that content identification is carried out on equipment logs through a preset multi-scale log semantic understanding device, a working state prediction result of each equipment in a preset time period is determined according to an identification result, a training data set of the multi-scale log semantic understanding device comprises a history equipment log and labels indicated by abstract texts corresponding to the history equipment log, the labels comprise normal labels representing the history equipment log as normal equipment logs and abnormal labels representing the history equipment log as abnormal equipment logs, a log abstract is generated according to syntactic dependency relationship and semantic information of the log texts, whether the equipment states corresponding to the logs are abnormal or not is determined through the log abstract, and further the state of the equipment in the preset time period is predicted according to the multi-scale log semantic understanding device, so that the purpose of determining the current state and the future state of the equipment without considering log formats is achieved, and the technical effect of accurately determining the equipment states is achieved, and the technical problem that the equipment states cannot be accurately determined in a log formatted scene due to high requirements on the log formats when the equipment states are judged according to the related technologies is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
Fig. 1 is a schematic structural view of a computer terminal (mobile terminal) according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for determining a device status according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a log summary generation process according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a log summary provided in accordance with an embodiment of the present community;
FIG. 5 is a schematic diagram of a training model of a multi-scale log semantic understanding device according to an embodiment of the present application;
FIG. 6 is an interface diagram of a device state prediction interface provided in accordance with an embodiment of the present application;
FIG. 7 is a flow chart of a device status determination procedure provided according to an embodiment of the present application;
Fig. 8 is a schematic structural diagram of an apparatus state determining device according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
With the continued development of networks and the increasing demand for digital services, networks have become more complex and critical. Modern society has been deeply affected by networks, which are used not only for daily communications, but also to support various business, educational, medical, entertainment, and other services. As a result, the importance of network operation becomes increasingly significant.
The core of network operation and maintenance is to ensure normal and safe operation of the network, and timely discovery of problems is of great importance. This critical task relies on continuous monitoring and deep analysis of weblogs, performance metrics, and security events. Efficient processing and prediction of these data sources has become an integral part of network operation. By comprehensively utilizing the data sources, a network operation and maintenance team can monitor network performance in real time, identify potential fault signs and rapidly cope with potential security threats, so that high availability and stability of the network are guaranteed. The comprehensive monitoring and data analysis also provides support for the early prediction of problems and the taking of preventive measures, helps to reduce the risk of potential faults, and makes network operation and maintenance more efficient and reliable.
At present, traditional log detection is based on rule matching, development system and deep learning model. The log detection method based on the rule matching and developing system needs accurate and complex rule definition and has extremely high requirement on the format of the log. This cannot effectively confirm the log content in a scene where the log content is complex and the log format is diversified, thereby determining whether the device is abnormal. And with the increase and change of the log, the rule needs to be updated continuously, and the maintenance cost is high. Development system-based methods require significant time and resources to build and maintain because custom detection logic, rules, and algorithms need to be written and maintained. The deep learning-based log detection requires a large number of data sets and labels to train the model, which results in poor versatility and failure determination in real time.
And the state of the equipment cannot be predicted according to the log in the related art.
In order to solve the above problems, related solutions are provided in the embodiments of the present application, and are described in detail below.
According to an embodiment of the present application, there is provided a method embodiment of a device state determination method, it being noted that the steps shown in the flowcharts of the drawings may be performed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.
The method embodiments provided by the embodiments of the present application may be performed in a mobile terminal, a computer terminal, or similar computing device. Fig. 1 shows a hardware block diagram of a computer terminal (or mobile device) for implementing a device state determination method. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …,102 n) processors 102 (the processors 102 may include, but are not limited to, a microprocessor MCU, a programmable logic device FPGA, etc. processing means), a memory 104 for storing data, and a transmission means 106 for communication functions. In addition, the method may further include: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power supply, and/or a camera. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 1 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
It should be noted that the one or more processors 102 and/or other data processing circuits described above may be referred to generally herein as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module, or incorporated, in whole or in part, into any of the other elements in the computer terminal 10 (or mobile device). As referred to in embodiments of the application, the data processing circuit acts as a processor control (e.g., selection of the path of the variable resistor termination connected to the interface).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the device state determining method in the embodiment of the present application, and the processor 102 executes the software programs and modules stored in the memory 104, thereby performing various functional applications and data processing, that is, implementing the device state determining method described above. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. The specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).
In the above operating environment, the embodiment of the present application provides a method for determining a device status, as shown in fig. 2, where the method includes the following steps:
step S202, obtaining device logs of all devices in a network system;
In the technical scheme provided in step S202, a dedicated log collecting device may be deployed on each device of the network system, so as to achieve efficient acquisition of device logs such as device operation logs and status logs. And the log is sent to a designated server or computer for analysis at intervals by a specific program.
Step S204, determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises keywords in the log text and scores of the keywords, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords;
In the technical solution provided in step S204, determining the abstract text of the device log according to the syntactic dependency relationship and semantic information of the log text in the device log and the preset log topic word stock includes: after preprocessing the device log, determining a directed acyclic graph according to the syntactic dependency relationship and semantic information of the log text in the preprocessed device log, wherein nodes in the directed acyclic graph represent log data words in the preprocessed log text, edges between the nodes represent association relationships between the log data words corresponding to the nodes, and the association relationships comprise at least one of the following: syntactic dependencies, semantic relationships; determining keywords in the preprocessed log text and scores of the keywords according to a preset log theme word stock and a directed acyclic graph; and generating abstract text according to the keywords and the scores.
As an alternative implementation manner, the edges in the directed acyclic graph are further used for indicating edge weight information, wherein the edge weight information is used for indicating the association degree between two log data words corresponding to the edges.
In some embodiments of the present application, determining keywords in the preprocessed log text according to the preset log topic word stock and the directed acyclic graph, and scoring the keywords includes: determining target log data words contained in a preset log theme word stock from all log data words indicated by the directed acyclic graph; determining target log data words as keywords, and increasing the weight of the keywords, wherein the weight is used for reflecting the importance degree of the keywords; and determining the score according to the weight of the keyword. The above-mentioned preset log subject words may be divided into abnormal subject words and state subject words, including but not limited to the words in the following table:
specifically, as shown in fig. 3, the generating flow of the abstract text of the device log includes the following steps:
Firstly, acquiring a log generated in the running process of equipment, preprocessing the equipment log, specifically processing modes including word segmentation, stop word removal and the like, and carrying out semantic analysis and syntactic dependency analysis on the preprocessed log so as to obtain a directed acyclic graph. Wherein, semantic analysis and syntactic dependency analysis can determine the association relationship among each word in the log,
The above syntactic dependencies may include master-predicate relationships, guest-move relationships, and dependencies, among others. Taking syntactic dependencies into account, semantic associations between words can be modeled more accurately, thereby improving the accuracy of the weights of edges in the graph.
And secondly, acquiring a log subject word stock, namely the preset log subject word stock. The log topic word library includes abnormal topic words and state topic words, such as down, error, discarded, BGP, ISIS, gigabit, ethernet, etc. The subject words in the log subject word library can be updated regularly, and the weight promotion degree corresponding to each subject word also needs to be updated in real time according to the importance of the updated subject word. The greater the importance, the greater the degree of weight lifting. In addition, the weight lifting degree can be a lifting proportion instead of a specific value, so that the situation that the weight value ranges in different equipment logs are possibly inconsistent can be adapted.
And thirdly, matching words corresponding to all nodes in the directed acyclic graph according to the log subject word stock, and determining keywords in the log subject word stock and corresponding weight promotion degree in the words corresponding to all the nodes. And then, the weight of the keyword in the set of phrase components corresponding to each node is promoted according to the weight promotion degree, and the weight reflects the importance of the keyword, namely the influence degree of the word on the final judgment result.
Specifically, the weight of each node may be determined according to the weight of the edge corresponding to the node. The more edges the node corresponds to, the greater the weight of the edges indicates that the importance of the node is higher, and the greater the weight of the node is. After determining the keywords, the weight of each keyword needs to be increased according to the weight increasing degree. And then normalizing the weights of all the keywords, and calculating the scores of the keywords. The normalized weight can be directly used as the score, or can be calculated according to a preset score calculation rule and the normalized weight. Wherein the relationship between the score and the weight is a positive correlation, that is, the greater the normalized weight, the higher the score.
A log summary as shown in fig. 4 may then be generated. It can be seen that the log summary includes a string of individual keywords and their scores.
By generating the log abstract in the method, log information of different types of equipment of different manufacturers can be summarized, and the method is more flexible and concise than the method for analyzing the log based on complex log analysis rules or systems in the related technology.
In addition, compared with the original Textrank algorithm, the improved Textrank algorithm shown in fig. 3 provided by the embodiment of the application introduces log subject terms to promote the weight of the subject terms. And, syntactic dependencies are utilized to provide more accurate and comprehensive log data semantic information, thereby improving the accuracy and reliability of detection of abnormal log data and enhancing the interpretability of the generated results.
The improved Textrank algorithm of the application comprises the following steps:
firstly, removing stop words;
secondly, obtaining word vectors of all the constituent words of each sentence in the log, and obtaining a combined vector of the sentence as a feature vector of the sentence;
thirdly, initializing a similarity matrix;
step four, extracting keywords;
fifthly, calculating the similarity between the keywords and filling a similarity matrix;
Step six, converting the similarity matrix into a graph structure;
and seventhly, sequencing and outputting the log abstract.
Step S206, determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log;
In the technical solution of step S206, after the log abstract is obtained, the importance degree of each keyword on the log semantics may be determined according to each keyword and score recorded in the log abstract, so as to determine whether the device state corresponding to the device log is an abnormal state according to the keywords.
As an alternative implementation manner, the device state corresponding to each keyword may be determined first, that is, it is determined that the device state corresponding to each keyword is normal or abnormal. For a keyword for which the corresponding device status is normal, a score corresponding to the keyword may be set to a positive value. For keywords for which the corresponding device state is abnormal, the score of the keyword may be set to be negative. And then adding the scores corresponding to all the keywords, if the final total score is greater than zero, the log can be considered normal, otherwise, the log is considered abnormal. And further processing is needed to be carried out on the logs which are considered to be normal through a preset multi-scale log semantic comprehension device to determine whether the device is abnormal in a preset time period. The preset time period may be a time period from the current time point, and the duration is a fixed duration. Or may be a predetermined period of time.
In addition to the above method of calculating the total score, it may be determined whether the device status is an abnormal status and a specific abnormal status type according to a preset mapping rule. For example, which keywords corresponding to various abnormal states are specific can be set, so that whether the keywords contained in the abstract correspond to the abnormal states and the types of the abnormal states, such as port abnormality, board abnormality, power supply abnormality and the like, are determined. And then, the possible abnormal state types can be arranged according to the scores of the keywords corresponding to the abnormal states, and abnormal alarm information is generated.
The occurrence frequency of each keyword can also be considered in the preset mapping rule. For example, only occasional concussions when ports concur down and up and finally showing up as normal, BGP neighbor failure belonging to anomaly, etc.
Step S208, carrying out content recognition on the equipment logs through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each equipment in a preset time period according to a recognition result, wherein a training dataset of the multi-scale log semantic understanding device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
In the technical scheme provided in step S208, content recognition is performed on the device log by a preset multi-scale log semantic understanding device, and determining, according to the recognition result, a working state prediction result of each device in a preset time period includes: processing the device log through the word segmentation model to obtain a word vector corresponding to the device log and a semantic attention weight value of the word vector, wherein the semantic attention weight value is used for reflecting the importance degree of the word vector; determining a global semantic understanding feature vector of the device log according to the word vector and the semantic attention weight value of the word vector; and processing the global semantic understanding feature vector through a preset multi-scale log semantic understanding device to obtain a classification result, wherein the classification result is a working state prediction result and is used for indicating the working state of equipment corresponding to the equipment log in a preset time period.
Specifically, before training a preset multi-scale log semantic understanding device or directly predicting the state of equipment by using the understanding device, the equipment log needs to be subjected to word segmentation processing through a word segmentation model. The word segmentation model can be obtained after fine adjustment on the basis of the BERT-Base model. The specific process of obtaining the word segmentation model and extracting the word vector according to the word segmentation model is as follows:
first, data preparation: sorting a log data corpus (such as a history equipment log of the last three years) within a certain period of time, and preprocessing data, including the steps of sentence segmentation, word segmentation, stop word removal and the like, so as to ensure the quality of input data;
secondly, performing preliminary word segmentation on the log by using WordPiece subword granularity, and dividing the word into subword units;
thirdly, constructing an input sequence:
Specifically, the format of the Bert requirement input sequence is a special tag [ CLS ], for classification tasks, and [ SEP ], for separating different sentences or text paragraphs. Therefore, the processed text needs to be converted into a Bert model input format;
Fourth, model pre-training:
the selected Bert model may be pre-trained using the prepared text data. The pre-training task typically includes masking language modeling and next sentence prediction;
Fifth, extracting word vectors: after the pre-training is completed, the hidden state of each word can be extracted from the Bert model as its word vector representation. In general, the hidden state corresponding to the [ CLS ] tag may be used as a vector representation of the entire text. The word segmentation model of the equipment log generated in the mode can effectively solve the problem of ambiguity of each word in the log, and the accuracy of a subsequent judging process is improved.
As an alternative embodiment, the word vector includes word granularity semantic connection feature vectors; processing the device log through the word segmentation model to obtain a word vector corresponding to the device log, wherein the semantic attention weight value of the word vector comprises the following steps: after word segmentation is carried out on the device logs through a word segmentation model, the device logs subjected to word segmentation are processed through a forward long-short-term memory network to obtain first-scale semantic coding feature vectors, and the device logs subjected to word segmentation are processed through a reverse long-short-term memory network to obtain second-scale semantic coding feature vectors; fusing the first scale word semantic coding feature vector and the second scale semantic coding feature vector to obtain a multi-scale semantic coding feature vector, wherein the multi-scale semantic coding feature vector comprises word granularity semantic connection feature vectors; and determining the weight value of each word granularity semantic connection feature vector relative to all word granularity semantic connection feature vectors as a semantic attention weight value.
In some embodiments of the application, determining a global semantic understanding feature vector for a device log from a word vector and a semantic attention weight value for the word vector comprises: determining a first sequence and a second sequence, wherein the first sequence comprises semantic attention weight values of semantic connection feature vectors with each word granularity, and the second sequence comprises semantic connection feature vectors with each word granularity; and fusing semantic connection feature vectors of each word granularity in the second sequence according to the first sequence to obtain a global semantic understanding feature vector.
Specifically, after the word segmentation model is obtained, a training model as shown in fig. 5 may be used for training, so as to obtain a preset multi-scale log semantic understanding device. The preset multi-scale log semantic comprehension device can comprise a forward LSTM (Long Short-Term Memory network), a reverse LSTM, a semantic attention module, a full connection layer and a classifier in FIG. 5. The specific training process is as follows:
firstly, acquiring historical log data (such as within one year) in a certain time, naming the log as date+equipment name, and acquiring a label corresponding to the log;
The labels of the logs may be determined according to the log digests of the logs in the manner of step S202-step S206.
Secondly, word segmentation is carried out on the history log data by using a word segmentation model;
Step three, performing word segmentation on the history log data, and then obtaining a sequence of semantic understanding feature vectors of the granularity of the history log data through a log data semantic encoder comprising BiLSTM networks, wherein the sequence of semantic understanding feature vectors of the granularity of the history log data is obtained through a forward LSTM network after word segmentation, and a second scale semantic encoding feature vector is obtained through a reverse LSTM network after word segmentation;
fourth, fusing the first scale code semantic coding feature vector and the second scale code semantic coding feature vector to obtain a multi-scale semantic coding feature vector;
Fifthly, for each historical log data word granularity semantic understanding feature vector in the historical log data word granularity semantic understanding feature vector sequence, calculating semantic attention weight values of each historical log data word granularity semantic understanding feature vector relative to all the historical log data word granularity semantic understanding feature vectors to obtain a semantic attention weight value sequence, wherein the sequence comprises weight values of each historical log data word granularity semantic understanding feature vector;
specifically, the formula for calculating the weight value is as follows:
Wherein h i is an i-th historical log data word granularity semantic understanding feature vector of the plurality of historical log data word granularity semantic understanding feature vectors, a and B are matrices of 1×n h, N h is a vector number of the plurality of historical log data word granularity semantic understanding feature vectors, N is a scale of the each historical log data word granularity semantic understanding feature vector, σ (·) is a Sigmoid function, and s i is an i-th semantic attention weight value of the plurality of semantic attention weight values.
Matrix a represents the weights or parameters associated with the i-th historical log data word granularity semantic understanding feature vector in the model, and matrix B represents the weights or parameters associated with all of the historical log data word granularity semantic understanding feature vectors.
The semantic features of important words in the log can be highlighted by determining the weight of each word vector, and the influence of the interference semantics of the useless words is reduced, so that the accuracy of the log abnormal prediction process is improved.
Step six, based on the sequence of the semantic attention weight values, fusing the sequence of the historical log data word granularity semantic understanding feature vectors to obtain log data global semantic understanding feature vectors;
Seventh, the log data global semantic understanding feature vector passes through a classifier to obtain a classification loss function value;
eighth, training the log data semantic encoder and the classifier including the word embedding model and BiLSTM model based on the classification loss function value and by back propagation of gradient descent.
It should be noted that the training process may be repeated at intervals (for example, repeated at intervals of seven days), so as to dynamically update the preset multi-scale log semantic comprehension device.
After training the preset multi-scale log semantic comprehension device, the specific process of predicting the equipment state through the preset multi-scale log semantic comprehension device comprises the following steps:
The first step: the logs and the labels in a period of time are obtained, the device class corresponding to each log is used as a primary folder such as CR, the device name is a secondary folder such as JS-NJ-GL-CR-1, the time is used as a tertiary folder such as 2023_11_21_04, and the last device name_catalog+label is used as a log name such as 2023_11_21_04_JS-NJ-GL-CR-1+anomaly. And sending the log data of the previous year into model training in a traversing mode, and updating a batch of data every preset time length.
Secondly, training a word segmentation model special for the equipment log;
the training method for training the segmentation model comprises the following steps:
1. reading file content of a device log;
2. preprocessing the read log, including sentence segmentation, word segmentation and stop word removal;
3. Taking BERT-Base as a model, and performing word segmentation by using a WordPiece tokenization alike tool;
4. Constructing an input sequence, and adding special marks [ CLS ] and [ SEP ] into the input sequence;
5. pre-training the word segmentation model according to the input sequence;
6. Extracting word vectors output by the word segmentation model;
specifically, when extracting word vectors, extracting the hidden state of each word from the word segmentation model, and extracting the hidden state corresponding to the [ CLS ] mark as the vector representation of the whole log text.
Thirdly, training a preset multi-scale log semantic understanding device.
The specific steps to be executed in the training process include: dividing the data set: importing data and dividing the data set according to the proportion. The proportion of the data set division can be determined according to the super parameter; introducing a log word segmentation model, and converting log data into word vectors; constructing a model and constructing a multi-scale log semantic understanding device model; model training: training the model by using a training set, and iterating for a plurality of rounds to improve the detection capability of the model on the abnormality; model evaluation: evaluating the performance of the model by using the test set, using the accuracy, recall, precision and F1 score as indexes, and improving the performance by adjusting the model structure and super parameters; continuously monitoring and updating: model performance is continuously monitored in the production environment, and continuous learning of the model is performed according to the new log data so as to adapt to the new abnormal mode and data distribution change.
In some embodiments of the present application, the device log to be predicted may be entered through an interface as shown in fig. 6 and the prediction results shown in fig. 6 may be obtained, or the model may be trained.
In summary, the device state determining flow provided by the embodiment of the present application is shown in fig. 7, and is divided into two parts, namely log detection and log prediction. The improved TextRank algorithm provided by the application is adopted in the log detection part to extract the log abstract and perform abnormality detection on the log, so as to determine whether the equipment is abnormal. And the log data processed by the TextRank algorithm is also sent to a database in the log prediction part as training data or further device state prediction is performed. In the log prediction part, log data can be processed through a log word segmentation model and a preset multi-scale log semantic comprehension device to obtain a prediction result, and the prediction result is used for indicating the equipment state of equipment in a preset time period.
Acquiring device logs of each device in the network system; determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises scores of keywords and keywords in the log text, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; the method comprises the steps that content identification is carried out on equipment logs through a preset multi-scale log semantic understanding device, a working state prediction result of each equipment in a preset time period is determined according to an identification result, a training data set of the multi-scale log semantic understanding device comprises a history equipment log and labels indicated by abstract texts corresponding to the history equipment log, the labels comprise normal labels representing the history equipment log as normal equipment logs and abnormal labels representing the history equipment log as abnormal equipment logs, a log abstract is generated according to syntactic dependency relationship and semantic information of the log texts, whether the equipment states corresponding to the logs are abnormal or not is determined through the log abstract, and further the state of the equipment in the preset time period is predicted according to the multi-scale log semantic understanding device, so that the purpose of determining the current state and the future state of the equipment without considering log formats is achieved, and the technical effect of accurately determining the equipment states is achieved, and the technical problem that the equipment states cannot be accurately determined in a log formatted scene due to high requirements on the log formats when the equipment states are judged according to the related technologies is solved.
In addition, the TextRank algorithm provided by the embodiment of the application can sort together logs of different types generated by different types of equipment in the form of text abstracts. Such as to put together an operation log and a status log obtained on different devices by log collection means. And the key word weight of the syntactic dependency and the log data is combined to promote, and the required abstract information is extracted from the log. The method is also a text summarization algorithm, and the algorithm summarizes the log summary by combining the weight of the subject words conforming to the network operation and maintenance, so that the effect of real-time processing is achieved, the response time is faster, the detection capability is stronger, and maintenance personnel can be helped to find problems in time.
In addition, the ambiguity is solved by using the word segmentation model using the exclusive log, the multi-scale log semantic understanding device is used for training, a more training understanding effect is obtained from the granularity of log data words, and the semantic attention is used for enabling the network training to have more emphasis. So that a powerful support can be provided for the detection and planning of communication network maintenance to better cope with uncertainties and variations.
The embodiment of the application provides a device state determining apparatus, fig. 8 is a schematic structural diagram of the apparatus, and as can be seen from fig. 8, the apparatus includes: a first processing module 80, configured to obtain a device log of each device in the network system; the second processing module 82 is configured to determine a summary text of the device log according to the syntactic dependency relationship and semantic information of the log text in the device log and a preset log topic word stock, where the summary text includes a keyword in the log text and a score of the keyword, and the summary text is used to indicate whether the device log is an abnormal log, and the score is used to represent an importance degree of the keyword; the third processing module 84 is configured to determine whether the device log is an abnormal log according to the summary text, and confirm that the device state of the device corresponding to the device log is an abnormal state if the device log is determined to be an abnormal log; the fourth processing module 86 is configured to identify content of the device log by using a preset multi-scale log semantic understanding device, and determine a working state prediction result of each device in a preset time period according to the identification result, where a training dataset of the multi-scale log semantic understanding device includes a history device log and a label indicated by a summary text corresponding to the history device log, and the label includes a normal label indicating that the history device log is a normal device log, and an abnormal label indicating that the history device log is an abnormal device log.
In some embodiments of the present application, the second processing module 82 determines the summary text of the device log according to the syntactic dependency and semantic information of the log text in the device log, and the preset log topic word stock includes: after preprocessing the device log, determining a directed acyclic graph according to the syntactic dependency relationship and semantic information of the log text in the preprocessed device log, wherein nodes in the directed acyclic graph represent log data words in the preprocessed log text, edges between the nodes represent association relationships between the log data words corresponding to the nodes, and the association relationships comprise at least one of the following: syntactic dependencies, semantic relationships; determining keywords in the preprocessed log text and scores of the keywords according to a preset log theme word stock and a directed acyclic graph; and generating abstract text according to the keywords and the scores.
In some embodiments of the present application, the edges in the directed acyclic graph are further used to indicate edge weight information, wherein the edge weight information is used to indicate a degree of association between two log data words corresponding to the edges.
In some embodiments of the present application, the second processing module 82 determines keywords in the preprocessed log text according to the preset log topic word library and the directed acyclic graph, and the scoring of the keywords includes: determining target log data words contained in a preset log theme word stock from all log data words indicated by the directed acyclic graph; determining target log data words as keywords, and increasing the weight of the keywords, wherein the weight is used for reflecting the importance degree of the keywords; and determining the score according to the weight of the keyword.
In some embodiments of the present application, the fourth processing module 86 performs content recognition on the device log through a preset multi-scale log semantic understanding device, and determining, according to the recognition result, a working state prediction result of each device in a preset time period includes: processing the device log through the word segmentation model to obtain a word vector corresponding to the device log and a semantic attention weight value of the word vector, wherein the semantic attention weight value is used for reflecting the importance degree of the word vector; determining a global semantic understanding feature vector of the device log according to the word vector and the semantic attention weight value of the word vector; and processing the global semantic understanding feature vector through a preset multi-scale log semantic understanding device to obtain a classification result, wherein the classification result is a working state prediction result and is used for indicating the working state of equipment corresponding to the equipment log in a preset time period.
In some embodiments of the application, the word vector comprises a word granularity semantic connection feature vector; the fourth processing module 86 processes the device log through the word segmentation model to obtain a word vector corresponding to the device log, and the semantic attention weight value of the word vector includes: after word segmentation is carried out on the device logs through a word segmentation model, the device logs subjected to word segmentation are processed through a forward long-short-term memory network to obtain first-scale semantic coding feature vectors, and the device logs subjected to word segmentation are processed through a reverse long-short-term memory network to obtain second-scale semantic coding feature vectors; fusing the first scale word semantic coding feature vector and the second scale semantic coding feature vector to obtain a multi-scale semantic coding feature vector, wherein the multi-scale semantic coding feature vector comprises word granularity semantic connection feature vectors; and determining the weight value of each word granularity semantic connection feature vector relative to all word granularity semantic connection feature vectors as a semantic attention weight value.
In some embodiments of the application, the fourth processing module 86 determining the global semantic understanding feature vector of the device log from the word vector and the semantic attention weight value of the word vector comprises: determining a first sequence and a second sequence, wherein the first sequence comprises semantic attention weight values of semantic connection feature vectors with each word granularity, and the second sequence comprises semantic connection feature vectors with each word granularity; and fusing semantic connection feature vectors of each word granularity in the second sequence according to the first sequence to obtain a global semantic understanding feature vector.
The respective modules in the device state determining apparatus may be program modules (for example, a set of program instructions for realizing a specific function), or may be hardware modules, and the latter may be expressed in the following form, but are not limited thereto: the expression forms of the modules are all a processor, or the functions of the modules are realized by one processor.
According to an embodiment of the present application, there is also provided a nonvolatile storage medium having a program stored therein, wherein when the program runs, a device in which the nonvolatile storage medium is controlled to execute the following device state determining method: acquiring device logs of all devices in a network system; determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises scores of keywords and keywords in the log text, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; and carrying out content recognition on the equipment logs through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each equipment in a preset time period according to a recognition result, wherein a training dataset of the multi-scale log semantic understanding device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
There is also provided, in accordance with an embodiment of the present application, an electronic device including: the device comprises a memory and a processor, wherein the processor is used for running a program stored in the memory, and the program runs to execute the following device state determining method: acquiring device logs of all devices in a network system; determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises scores of keywords and keywords in the log text, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; and carrying out content recognition on the equipment logs through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each equipment in a preset time period according to a recognition result, wherein a training dataset of the multi-scale log semantic understanding device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
According to another aspect of embodiments of the present application, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements a device state determination method of: acquiring device logs of all devices in a network system; determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises scores of keywords and keywords in the log text, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords; determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of the equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log; and carrying out content recognition on the equipment logs through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each equipment in a preset time period according to a recognition result, wherein a training dataset of the multi-scale log semantic understanding device comprises the historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the related art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.
Claims (11)
1. A device state determination method, comprising:
acquiring device logs of all devices in a network system;
Determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises keywords in the log text and scores of the keywords, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords;
Determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log;
And carrying out content recognition on the equipment logs through a preset multi-scale log semantic understanding device, and determining a working state prediction result of each equipment in a preset time period according to a recognition result, wherein a training dataset of the multi-scale log semantic understanding device comprises historical equipment logs and labels indicated by abstract texts corresponding to the historical equipment logs, and the labels comprise normal labels representing the historical equipment logs as normal equipment logs and abnormal labels representing the historical equipment logs as abnormal equipment logs.
2. The apparatus state determining method according to claim 1, wherein determining the digest text of the apparatus log according to the syntactic dependency and semantic information of the log text in the apparatus log and a preset log topic word stock comprises:
After preprocessing the device log, determining a directed acyclic graph according to the syntactic dependency relationship and semantic information of the log text in the preprocessed device log, wherein nodes in the directed acyclic graph represent log data words in the preprocessed log text, edges between the nodes represent association relationships between the log data words corresponding to the nodes, and the association relationships comprise at least one of the following: syntactic dependencies, semantic relationships;
Determining the keywords in the preprocessed log text and the scores of the keywords according to the preset log theme word stock and the directed acyclic graph;
and generating the abstract text according to the keywords and the scores.
3. The apparatus state determining method according to claim 2, wherein edges in the directed acyclic graph are further used to indicate edge weight information, wherein the edge weight information is used to indicate a degree of association between two log data words corresponding to the edges.
4. The apparatus state determination method according to claim 2, wherein determining the keywords in the log text after preprocessing from the preset log topic word stock and the directed acyclic graph, and the scoring of the keywords includes:
Determining target log data words contained in the preset log topic word library from all the log data words indicated by the directed acyclic graph;
Determining the target log data word as the key word, and increasing the weight of the key word, wherein the weight is used for reflecting the importance degree of the key word;
and determining the score according to the weight of the keyword.
5. The device state determining method according to claim 1, wherein the content recognition of the device log by a preset multi-scale log semantic understand device, and determining the working state prediction result of each device in a preset time period according to the recognition result comprises:
Processing the device log through a word segmentation model to obtain a word vector corresponding to the device log and a semantic attention weight value of the word vector, wherein the semantic attention weight value is used for reflecting the importance degree of the word vector;
determining a global semantic understanding feature vector of the device log according to the word vector and the semantic attention weight value of the word vector;
And processing the global semantic understanding feature vector through the preset multi-scale log semantic understanding device to obtain a classification result, wherein the classification result is the working state prediction result and is used for indicating the working state of equipment corresponding to the equipment log in the preset time period.
6. The device state determination method of claim 5, wherein the word vector comprises a word granularity semantic connection feature vector; processing the device log through a word segmentation model to obtain a word vector corresponding to the device log, wherein the semantic attention weight value of the word vector comprises:
after word segmentation is carried out on the equipment logs through a word segmentation model, the equipment logs subjected to word segmentation are processed through a forward long-short-term memory network to obtain first-scale semantic coding feature vectors, and the equipment logs subjected to word segmentation are processed through a reverse long-short-term memory network to obtain second-scale semantic coding feature vectors;
Fusing the first scale word semantic coding feature vector and the second scale semantic coding feature vector to obtain a multi-scale semantic coding feature vector, wherein the multi-scale semantic coding feature vector comprises the word granularity semantic connection feature vector;
And determining the weight value of each word granularity semantic connection feature vector relative to all word granularity semantic connection feature vectors as the semantic attention weight value.
7. The device state determination method of claim 6, wherein determining a global semantic understanding feature vector of the device log from the word vector and the semantic attention weight value of the word vector comprises:
Determining a first sequence and a second sequence, wherein the first sequence comprises the semantic attention weight value of each word granularity semantic connection feature vector, and the second sequence comprises each word granularity semantic connection feature vector;
And fusing the semantic connection feature vectors of the word granularity in the second sequence according to the first sequence to obtain the global semantic understanding feature vector.
8. A device state determining apparatus, characterized by comprising:
The first processing module is used for acquiring device logs of all devices in the network system;
The second processing module is used for determining abstract text of the equipment log according to syntactic dependency relationship and semantic information of log text in the equipment log and a preset log topic word stock, wherein the abstract text comprises keywords in the log text and scores of the keywords, the abstract text is used for indicating whether the equipment log is an abnormal log or not, and the scores are used for reflecting importance degrees of the keywords;
The third processing module is used for determining whether the equipment log is an abnormal log according to the abstract text, and confirming that the equipment state of equipment corresponding to the equipment log is an abnormal state under the condition that the equipment log is determined to be the abnormal log;
The fourth processing module is configured to identify content of the device log through a preset multi-scale log semantic understand device, and determine a working state prediction result of each device in a preset time period according to an identification result, where a training dataset of the multi-scale log semantic understand device includes a historical device log and a tag indicated by a summary text corresponding to the historical device log, and the tag includes a normal tag representing that the historical device log is a normal device log, and an abnormal tag representing that the historical device log is an abnormal device log.
9. A nonvolatile storage medium, wherein a program is stored in the nonvolatile storage medium, and wherein the program, when executed, controls a device in which the nonvolatile storage medium is located to execute the device state determining method according to any one of claims 1 to 7.
10. An electronic device, comprising: a memory and a processor for executing a program stored in the memory, wherein the program executes the device state determination method according to any one of claims 1 to 7.
11. A computer program product comprising a computer program which, when executed by a processor, implements the device state determination method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410418453.XA CN118312383A (en) | 2024-04-08 | 2024-04-08 | Device state determining method and device, nonvolatile storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410418453.XA CN118312383A (en) | 2024-04-08 | 2024-04-08 | Device state determining method and device, nonvolatile storage medium and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118312383A true CN118312383A (en) | 2024-07-09 |
Family
ID=91721775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410418453.XA Pending CN118312383A (en) | 2024-04-08 | 2024-04-08 | Device state determining method and device, nonvolatile storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118312383A (en) |
-
2024
- 2024-04-08 CN CN202410418453.XA patent/CN118312383A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113094200B (en) | Application program fault prediction method and device | |
| CN109598517B (en) | Commodity clearance processing, object processing and category prediction method and device thereof | |
| US20220100772A1 (en) | Context-sensitive linking of entities to private databases | |
| CN109086265B (en) | Semantic training method and multi-semantic word disambiguation method in short text | |
| CN111814482B (en) | Text key data extraction method and system and computer equipment | |
| CN111177367B (en) | Case classification method, classification model training method and related products | |
| US20220100967A1 (en) | Lifecycle management for customized natural language processing | |
| CN110334186A (en) | Data query method, apparatus, computer equipment and computer readable storage medium | |
| CN113590824A (en) | Method and device for constructing causal graph and related equipment | |
| CN112036185B (en) | Method and device for constructing named entity recognition model based on industrial enterprise | |
| CN114528845A (en) | Abnormal log analysis method and device and electronic equipment | |
| JP6709812B2 (en) | Relationship estimation model learning device, method, and program | |
| CN112685374B (en) | Log classification method and device and electronic equipment | |
| CN115359799A (en) | Speech recognition method, training method, device, electronic equipment and storage medium | |
| EP4222635A1 (en) | Lifecycle management for customized natural language processing | |
| CN111967253A (en) | Entity disambiguation method and device, computer equipment and storage medium | |
| CN115374711B (en) | Service life prediction method of rotating multi-component system and related device | |
| CN114969334B (en) | Abnormal log detection method and device, electronic equipment and readable storage medium | |
| CN116701752A (en) | News recommendation method and device based on artificial intelligence, electronic equipment and medium | |
| CN116795978A (en) | Complaint information processing method and device, electronic equipment and medium | |
| CN118312383A (en) | Device state determining method and device, nonvolatile storage medium and electronic device | |
| CN115238093A (en) | Model training method and device, electronic equipment and storage medium | |
| CN115618264A (en) | Method, apparatus, device and medium for topic classification of data assets | |
| CN113779256A (en) | File auditing method and system | |
| CN117852553B (en) | Language processing system for extracting component transaction scene information based on chat record |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |