WO2020168676A1 - Method for constructing network fault handling model, fault handling method and system - Google Patents
Method for constructing network fault handling model, fault handling method and system Download PDFInfo
- Publication number
- WO2020168676A1 WO2020168676A1 PCT/CN2019/096623 CN2019096623W WO2020168676A1 WO 2020168676 A1 WO2020168676 A1 WO 2020168676A1 CN 2019096623 W CN2019096623 W CN 2019096623W WO 2020168676 A1 WO2020168676 A1 WO 2020168676A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- network
- model
- data
- network fault
- target
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000003062 neural network model Methods 0.000 claims abstract description 71
- 239000013598 vector Substances 0.000 claims description 141
- 238000012545 processing Methods 0.000 claims description 77
- 238000010276 construction Methods 0.000 claims description 38
- 210000002569 neuron Anatomy 0.000 claims description 16
- 238000003672 processing method Methods 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 5
- 230000003287 optical effect Effects 0.000 abstract description 26
- 238000013526 transfer learning Methods 0.000 abstract description 5
- 238000004891 communication Methods 0.000 abstract description 4
- 239000011159 matrix material Substances 0.000 description 74
- 230000006870 function Effects 0.000 description 21
- 230000008569 process Effects 0.000 description 17
- 238000007726 management method Methods 0.000 description 15
- 238000013480 data collection Methods 0.000 description 7
- 238000013508 migration Methods 0.000 description 7
- 230000005012 migration Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 5
- 230000008439 repair process Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000002776 aggregation Effects 0.000 description 3
- 238000004220 aggregation Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000002542 deteriorative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0681—Configuration of triggering conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
Definitions
- the invention relates to the field of communication technology, in particular to a method for constructing a network fault processing model, a fault processing method and a system.
- optical network systems include wireless networks, access networks, bearer networks, and data centers with different fault characteristics.
- the wireless network, access network, bearer network, and data center are separately established.
- the machine learning model leads to the problem of repeated learning; on the other hand, some target fields have incomplete sample data and it is difficult to establish an effective machine learning model.
- the purpose of the embodiments of the present invention is to provide a method for constructing a network fault processing model, a fault processing method and a system, based on the deep neural network model of the source field in the optical network, and through cross-domain migration learning, the network fault processing model of the target field is obtained. .
- an embodiment of the present invention provides a method for constructing a network fault handling model, which includes:
- the sample set of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data;
- the network fault handling model of the target domain is constructed based on the deep neural network model of the source domain.
- the deep neural network model of the source domain is used as the network fault handling model of the target domain; or,
- the difference set between the sample set of the target domain and the sample set of the source domain is obtained, and the network fault handling model of the target domain is optimized based on the difference set .
- a second input vector and a corresponding second output vector are extracted from the difference set, and the network fault handling model in the target domain is retrained.
- the network fault handling model in the target field is retrained.
- the weight coefficient of the neuron function of the network fault handling model of the target field is corrected based on the difference set to obtain an optimized network fault handling model of the target field.
- the input vector of the deep neural network model of the source domain includes the quantized alarm data and fault data
- the output vector is the quantized processing Configuration data
- an embodiment of the present invention provides a network fault processing method, which includes:
- the output vector of the network fault handling model is delivered to the relevant equipment of the target network.
- an embodiment of the present invention provides a construction system for a network fault handling model, which includes:
- An acquisition module which is used to acquire or establish a deep neural network model of the source domain based on the sample set of the source domain in the network;
- a processing module which is used to establish a sample set of the target field in the network, the sample set of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data; and calculate the target field and the source The coincidence rate of the sample set of the field;
- the construction module is used to construct a network fault handling model of the target domain based on the deep neural network model of the source domain when the coincidence rate of the sample set of the target domain and the source domain reaches a set threshold.
- the construction module is used to use the deep neural network model of the source domain as the network fault handling model of the target domain;
- the first input vector and the corresponding first output vector are concentratedly extracted, and the deep neural network model of the source domain is retrained to obtain the network fault handling model of the target domain.
- the processing module is further configured to obtain the difference between the sample set of the target field and the sample set of the source field;
- the construction module is also used for optimizing the network fault processing model of the target domain based on the difference set.
- the construction module is configured to extract a second input vector and a corresponding second output vector from the difference set, and retrain the network fault handling model of the target domain.
- the construction module is used to extract a third input vector from the difference set, and input it into the network fault processing model of the target field to obtain a third output vector; After the feedback result corrects the third input vector and the third output vector, the network fault handling model in the target field is retrained.
- the construction module is used to modify the weight coefficients of the neuron function of the network fault handling model of the target field based on the difference set to obtain an optimized value of the target field Network fault handling model.
- the input vector of the deep neural network model of the source domain includes the quantized alarm data and fault data
- the output vector is the quantized processing Configuration data
- an embodiment of the present invention provides a network fault processing system, which includes:
- Input control module which is used to obtain alarm data and fault data of the target network and perform quantitative processing
- the model processing module is used to store the network fault processing model constructed by the network fault processing model construction system described in the third aspect, and input the quantitatively processed alarm data and fault data into the network fault processing model to obtain State the output vector of the network fault handling model;
- the output control module is used to deliver the output vector of the network fault handling model to the relevant equipment of the target network.
- the embodiment of the present invention uses a method for constructing a network fault handling model to obtain or establish a deep neural network model of the source field based on a sample set of the source field in the network; establish a sample set of the target field in the network,
- the sample sets of the target field and the source field both include quantified alarm data, fault data, and configuration data, and have an intersection; when the coincidence rate of the sample sets of the target field and the source field reaches the set threshold, the source field-based Deep neural network model to build a network fault handling model in the target field.
- the network fault handling model of the target domain is obtained.
- Figure 1 is a schematic diagram of a cloud network architecture
- FIG. 2 is a flowchart of a method for constructing a network fault handling model according to an embodiment of the present invention
- Figure 3 is a schematic diagram of obtaining data from a database and performing vectorization and matrixization
- FIG. 4 is a flowchart of a method for constructing a network fault handling model according to another embodiment of the present invention.
- Figure 5 is an example of a multi-layer high-dimensional space
- Figure 6 is an example of the construction and optimization of the network fault handling model in the target field
- FIG. 7 is a schematic diagram of a construction system of a network fault handling model according to an embodiment of the present invention.
- Fig. 8 is a schematic diagram of a network fault handling system according to an embodiment of the present invention.
- the network involved in the embodiment of the present invention may be a traditional optical transport network such as (Optical Transport Network, OTN), Packet Transport Network (PTN), and Packet Optical Transport Network (Packet Optical Transport Network, POTN), or It is a cloud network.
- OTN Optical Transport Network
- PTN Packet Transport Network
- POTN Packet Optical Transport Network
- Figure 1 is a schematic diagram of a cloudized network architecture.
- the bottom left part of Figure 1 is a cloudized network base station, including Active Antenna Unit (AAU), Centralized Unit (CU), and Distributed Unit (Distributed Unit, DU), where CU supports non-real-time wireless high-level protocols and some core network sink functions and edge application functions, and DU supports physical layer functions and real-time functions.
- the lower part of Figure 1 is the cloudized network access ring, aggregation ring and core ring.
- the alarm data, fault data and configuration data of the network devices in these ring networks are reported to the edge data in the upper part of Figure 1 through the network management platform or the controller platform.
- the core network functions of the 5G core network are divided into user plane (UP) functions and control plane (CP) functions.
- UP user plane
- CP control plane
- these data centers are responsible for the management, orchestration, and control of cloud-based networks.
- UP user plane
- CP control plane
- these data centers are responsible for the management, orchestration, and control of cloud-based networks.
- they deploy intelligent platforms for cloud-based networks, and build a cloud-based network operation and maintenance management knowledge base based on massive network data and powerful computing capabilities. , As the brain of the cloud network.
- the data center first cleans the data to remove redundant, low-quality data and obtain high-quality alarm data Sets, fault data sets, and configuration data sets are stored in the database.
- the source domain may be defined as an access network
- the target domain may be defined as an aggregation network
- the source domain may be defined as a core network
- the target domain may be defined as a data center network, which is not limited.
- Devices on different networks of clouded networks may have their own professional network management or dedicated control platforms.
- the source domain and the target domain may also be an access network, an aggregation network, and a core network in a traditional optical network (OTN, PTN, and POTN), respectively.
- the network equipment reports the alarm data and related fault data to the network management platform, and the network management platform submits it to the data center.
- Alarms generated by network equipment include root cause alarms and derivative alarms, and there is a correlation between root cause alarms and derivative alarms.
- the embodiment of the present invention is based on the deep neural network model of the source domain in the network, and obtains the network fault processing model of the target domain through cross-domain migration learning. Therefore, when an alarm or failure occurs in the target field, the network fault handling model in the target field automatically generates configuration data, and distributes the equipment in the target field through the management control platform to complete the equipment recovery, switching, parameter adjustment and rerouting of the target field, etc. Operate to achieve self-healing of network failures in the target area.
- the embodiment of the present invention solves the problem of repeated learning when establishing network fault handling models in different fields in the network, and the problem of incomplete sample data in some target fields, which makes it difficult to establish an effective machine learning model, and is conducive to unifying different fields in the network management.
- Figure 2 shows a flowchart of a method for constructing a network fault handling model according to an embodiment of the present invention.
- the network includes a source field and a target field.
- the method for constructing a network fault handling model includes:
- S110 obtains the sample set of the source domain and its deep neural network model.
- S120 establishes a sample set of the target field.
- the sample sets of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data.
- step S110 the deep neural network model of the source domain is created in advance based on the sample set of the source domain.
- Common deep neural network models include Stacked Auto-Encoder, Convolutional Neural Network (CNN), Deep Belief Network, etc.
- the sample set in the source domain includes quantitatively processed alarm data, fault data, and configuration data, see the specific description in step S120.
- the input and output sample data of the deep neural network model in the source field usually takes the form of vectors, that is, the sample set includes the alarm data vector group, the fault data vector group and the configuration data respectively obtained according to the alarm data, fault data and configuration data of the source field Vector set.
- the input vector of the deep neural network model of the source domain includes quantized alarm data and fault data
- the output vector is quantized configuration data
- the quantified alarm data and fault data are used as input, and the quantized configuration data is used as the output.
- a deep neural network model is generated and trained. Through large-scale high-quality sample data training, the depth The neural network model learns the fault intelligent self-healing knowledge in the source field, and the relevant knowledge is stored in a series of neurons in the deep neural network in an abstract form. Through the deep neural network model in the source domain, the association rules between the optical network-derived alarms and the root-cause alarms are excavated, and the precise relationship between the root-cause alarms and the fault location is generated.
- the network configuration plan can be given according to the alarm and fault information, and the network management and The controller platform realizes the automatic repair of faults in the optical network source field.
- step S120 the alarm data, fault data and configuration data at multiple time points are obtained from the database of the target field, and the sample set of the target field is obtained after quantization processing.
- the field definitions of the alarm data, fault data, and configuration data of the source field and the target field are the same, but the ordering is not required to be the same.
- all the alarm data, fault data and configuration data of the target field in the set time period can be obtained from the database, or by day, week or month Periodically obtain all alarm data, fault data and configuration data of the target field from the database.
- the set time period or cycle includes alarm data at multiple time points, fault data at multiple time points, and configuration data at multiple time points.
- Alarm data, fault data, and configuration data are not only heterogeneous data, but these data include various types of fields, and different fields have different dimensions. Therefore, the quantization process includes the vectorized representation of heterogeneous data of different dimensions, including:
- each piece of alarm data, fault data or configuration data is converted into a basic vector V b , and each element of the basic vector V b is the value of a field in each piece of alarm data, fault data or configuration data.
- the sample data set acquired all alarms are constituted M a in a trap data, wherein the alerting data generated at a time point may be one or more pieces, each alarm data field has N a.
- a piece of alarm data shown in Figure 3 includes eight fields, namely: the sequence number of the alarm data Seq.No., address Addr., line number Line, alarm type AlarmType, alarm start time BeginTime, alarm end Time EndTime, board type BoardType, and network element type NetType, where the alarm start time BeginTime and alarm end time EndTime are accurate to seconds, the address Addr. and alarm type AlarmType are character numbers, and the network element type NetType is an integer value.
- the values of all fields of the alarm data shown in Fig. 3 are converted into real numbers, and thus expressed as elements of a vector.
- the integer values of these fields are represented in the vector as element values.
- the minimum value of all alarm start time BeginTime and alarm end time EndTime can be corresponding to the value 1, and the number of seconds between other times and the minimum time can be added to the value 1, and the corresponding values of the alarm start time BeginTime and the alarm end time EndTime can be obtained respectively .
- the alarm start time BeginTime is 10 seconds longer than the minimum time
- the alarm start time BeginTime corresponds to the value 11.
- the two fields are arranged in lexicographic order, and then numbered from 1, and the string is converted to a value as an element of the vector .
- S122 performs dimension conversion on the basis vector V b , and the converted vector V is the hadamard product of the basis vector V b and the dimension expansion vector V s , namely
- the element of the dimension expansion vector V s is the expansion or reduction multiple of the corresponding element of the basic vector V b . For example, if the bandwidth unit M is expanded to giga G, the element of the dimension expansion vector V s is 1024.
- the basic vector can be multiplied by the corresponding elements of the dimension expansion vector to generate sample data suitable for the training requirements.
- the configuration data and fault data in the lower left part of Fig. 3 are also converted into corresponding vectors.
- the configuration data includes Num_CPUs: 4, which is the number of CPU cores.
- the vector group in the lower part of Fig. 3 shows two vectors. The alarm data and configuration data are converted.
- the above method can also be used to construct the data basis vector and the dimension expansion vector.
- the number of key/value pairs in XML corresponds to the number of the vector Dimension
- the value of the vector element corresponds to the Value value in the XML document.
- Construct three pairs of vector groups for the target field which are the basic vector group of alarm data and the expanded vector group of dimensions, the basic vector group of fault data and the expanded vector group of dimensions, and the basic vector group of configuration data and the expanded vector group of dimensions.
- alarm data vectors obtained by the data converting M a M a in a trap number alarm data vectors, each data vector having N a warning elements
- vector data group includes fault fault M f M f obtained from the data conversion section fault data vectors, each data vector having N f fault elements
- vector set of configuration data including configuration data obtained by conversion of configuration data vector M c M c from the bar, each configuration data vector with N c elements.
- the alarm data vector group, the fault data vector group, and the configuration data vector group can also be expressed in matrix.
- the alarm data vector group is stored in a two-dimensional empty matrix in the form of row vectors to form an alarm matrix.
- M a 7000 if there is data in a trap
- trap matrix is formed 7000 rows and 8 columns.
- fault matrix and configuration matrix can be constructed.
- S123 finds the intersection of the sample set of the target field and the source field.
- the intersection of the sample sets of the target domain and the source domain is obtained.
- step S130 when the coincidence rate of the sample set of the source field and the sample set of the target field reaches the set threshold, the network fault handling model of the target field can be constructed in different implementation manners, for example, one of the following implementation manners can be adopted One:
- Implementation mode 1 The deep neural network model in the source domain is used as the network fault processing model in the target domain.
- Embodiment 2 Extract the first input vector and the corresponding first output vector from the intersection, retrain the deep neural network model of the source domain, and obtain the network fault handling model of the target domain.
- the network fault handling model of the target domain is the same as that of the source domain.
- the deep neural network model is similar to the deep neural network model.
- the source domain's deep neural network model After the source domain's deep neural network model obtains the source domain fault self-healing knowledge base, the intersection of the sample data of the source domain and the target domain is obtained. Because the intersection fault self-healing knowledge is already included in the source domain fault self-healing knowledge base Therefore, based on the intersection of the sample data of the source domain and the target domain, the knowledge base for self-healing faults in the source domain is migrated to the target domain to realize cross-domain migration learning. In the process of transfer learning, if the intersection of the sample data in the source field and the target field is relatively large (that is, the coincidence rate is higher), the transfer learning effect will be better.
- the size of the threshold can be adjusted according to specific scenarios.
- the threshold is a percentage value.
- the overlap rate of the data intersection of the source domain and the target domain is between 0% and 100%.
- a coincidence rate of 60% means that the sample data of the source field and the target field are 60% the same, and 40% are different.
- the threshold is small, the migration process of the knowledge base of the source domain fault self-healing is faster, and the subsequent correction and optimization process of the weight parameter is longer. Conversely, if the threshold is larger, the migration process of the knowledge base for self-healing failures in the source domain is slower, but the subsequent correction and optimization process of the weight parameters is shorter.
- the coincidence rate is lower than the set threshold, you need to add new data to the sample set of the target field, or you can select a batch of data samples again to supplement the intersection data of the source field and the target field, until the coincidence rate exceeds the set Threshold.
- steps S110 and S120 are executed in sequence, and in another embodiment of the present invention, steps S110 and S120 can also be executed in other ways, for example, obtaining alarm data and fault data in the source domain and target domain respectively After quantitative processing, the sample sets of the source field and the target field are established respectively, and then the deep neural network model of the source field is constructed.
- Fig. 4 is a flowchart of a method for constructing a network fault handling model according to another embodiment of the present invention.
- the method for constructing a network fault handling model includes:
- S200 data collection and preprocessing It specifically includes:
- the data collection and preprocessing process of the source field and target field are basically the same.
- the alarm data, fault data and configuration data of the optical network are uploaded to the three types of data centers by the network management platform or the controller platform. Because the alarm data, fault data, and configuration data of the massive optical network contain a large amount of redundant, incomplete, and inconsistent data, the three types of data centers will first clean the data, remove the redundant, low-quality data, and get the highest Quality alarms, faults, and configuration data sets are stored in the source domain database and target domain database respectively.
- S210 constructs a deep neural network model of the source domain.
- the method of constructing deep neural network models in the source domain is not limited.
- common deep neural network models include Stacked Auto-Encoder, Convolutional Neural Network (CNN), and Deep Belief Network ( Deep Belief Network) and so on.
- Step S220 specifically includes:
- a multi-layer high-dimensional space is constructed to realize the unified representation of alarm data, fault data, and configuration data in the source and target fields.
- the vectorization and matrix representation methods of heterogeneous data of different dimensions are successively used to convert the alarm data, fault data and configuration data of the source and target fields into one-dimensional vectors, and then respectively express them into corresponding two-dimensional matrices. It specifically includes: the construction process of one-dimensional vector and the construction process of two-dimensional matrix.
- a two-dimensional alarm matrix, a fault matrix, and a configuration matrix are constructed according to the alarm data, fault data, and configuration data of the source field, and a two-dimensional alarm matrix, fault, and configuration data are constructed according to the alarm data, fault data, and configuration data of the target field.
- the construction methods of the matrix and the configuration matrix, the one-dimensional vector and the two-dimensional matrix are similar to the foregoing embodiments, and will not be repeated here.
- the number of rows and columns of the two-dimensional matrix may be different, as shown in Table 1:
- Table 1 Examples of the number of rows and columns of the two-dimensional matrix of the source and target fields
- Matrix type Number of rows and columns of the alarm matrix Number of rows and columns of the fault matrix Configure the number of rows and columns of the matrix Source field 5000 ⁇ 12 7000 ⁇ 18 3000 ⁇ 32 Target field 3000 ⁇ 8 5000 ⁇ 12 2000 ⁇ 35
- the maximum number of rows and the maximum number of columns of all alarm matrix, fault matrix, and configuration matrix use the maximum number of rows and the maximum number of columns as the number of rows and columns of each layer of the two-dimensional matrix in the multi-layer high-dimensional space.
- the number of rows and columns of each layer of a two-dimensional matrix in a multi-layer high-dimensional space are 7000 and 35, respectively.
- the number of rows 7000 means that the largest number of rows in the six matrices is the number of rows of the source field fault matrix
- the number of columns 35 means that the largest number of columns in the six matrices is the number of columns of the target field configuration matrix.
- a 6-layer high-dimensional space representation model is constructed based on the six matrices in Table 1 above, and 6 empty matrices with 7000 rows and 35 columns are generated, and these 6 The data in the matrix is copied to the newly generated empty matrix, and the matrix elements without stored data are filled with zero elements.
- the multi-layer high-dimensional space constructed for the source field and the target field is shown in Figure 5.
- the six-layer multi-layer high-dimensional space D R(K 1 , K 2 , K 3 ), the first to the third
- the layers are the alarm data layer, fault data layer and configuration data layer of the source field, corresponding to the alarm matrix, fault matrix and configuration matrix of the source field respectively.
- the fourth to sixth layers are the alarm data layer, fault data layer and
- the configuration data layer corresponds to the alarm matrix, fault matrix and configuration matrix of the target field.
- a multi-layer high-dimensional space for multiple fields, such as an access network, a convergence network, a core network, and a data center network, which is not limited.
- the vectorization and matrix representation methods for heterogeneous data of different dimensions can convert structured and semi-structured optical network data of different dimensions into vectors and matrices, because there are a large number of zero elements
- multi-layer high-dimensional space is a sparse matrix.
- the classic sparse matrix storage method can be used to save data to save storage space.
- constructing a multi-layer high-dimensional space not only realizes the unified representation of sample data in the source domain and the target domain, but also realizes the intercommunication and sharing of cross-domain sample data from different vendors, removing information island barriers for subsequent machine learning.
- the subspace includes at least one submatrix of an alarm data layer, a fault data layer, and a configuration data layer.
- the sub-matrix can be a sub-matrix in one layer of the multi-layer high-dimensional space; the sub-matrix can also be two or more layers of the multi-layer high-dimensional space, where each layer of the sub-matrix is a layer of the multi-layer high-dimensional space A sub-matrix of.
- the matrix S and the matrix T in FIG. 6 respectively represent the sample set of the source field and the sample set of the target field in the transfer learning process, and these two matrices are both matrices with 3 rows and 3 columns.
- S222 finds the intersection of the sample set of the target domain and the source domain.
- intersection is also a subspace of a multi-layer high-dimensional space, and the subspace includes at least one submatrix of an alarm data layer, a fault data layer, and a configuration data layer.
- the sub-matrix can be a sub-matrix in one layer of the multi-layer high-dimensional space; the sub-matrix can also be two or more layers of the multi-layer high-dimensional space, where each layer of the sub-matrix is a layer of the multi-layer high-dimensional space A sub-matrix of.
- the matrix S and the matrix T respectively represent the sample set of the source field and the sample set of the target field in the transfer learning process. Both matrices are three-row and three-column matrices.
- Step S223 is basically the same as step S130 in the foregoing embodiment.
- the overlap between the sample set of the source field and the sample set of the target field is the intersection.
- the deep neural network model of the source field can be directly used as the network fault processing model of the target field, or the first input vector and the corresponding can be extracted from the intersection. Retrain the deep neural network model of the source domain to obtain the network fault handling model of the target domain, thereby migrating the fault handling knowledge base of the source domain to the fault handling knowledge base of the target domain.
- the matrices S and T are 9-element matrices, and the intersection matrix I is a 6-element matrix. If the threshold is set to 60%, the proportion of the intersection data exceeds the set Set a threshold of 60%, you can directly use the deep neural network model in the source field as the network fault handling model in the target field, or extract the first input vector and the corresponding first output vector from the intersection to retrain the deep neural network in the source field Network model to obtain the network fault handling model of the target field.
- the method for constructing a network fault handling model further includes: S300 obtains the difference set between the sample set of the target domain and the sample set of the source domain, and optimizes the network fault handling model of the target domain based on the difference set.
- the second input vector and the corresponding second output vector can be extracted from the difference set, and the network fault handling model of the target domain can be retrained.
- the third input vector can also be extracted from the difference set, and the third output vector can be obtained by inputting the network fault processing model of the target field; the third input vector and the third output vector are corrected according to the expert evaluation feedback result , Retrain the network fault handling model in the target domain.
- retraining the network fault handling model of the target field includes: correcting the weight coefficient of the neuron function of the network fault handling model of the target field based on the difference set to obtain an optimized network fault handling model of the target field.
- the intersection matrix I in Figure 6 is used to directly generate the weight parameters of the fitting function in the deep neural network model of the target field.
- the lower part of Figure 6 is the difference matrix D between the source field and the target field.
- the difference matrix D is a 2 row 3
- the column matrix and the difference matrix D are used to optimize the weight parameters of the fitting function in the deep neural network model of the target field.
- FIG. 6 Take FIG. 6 as an example to illustrate the process of modifying the weight coefficient w 22 of the neuron function f 22 through the difference set data x 22 and y 22 .
- the configuration data is the configuration plan quantified representation data to construct the output vector.
- the quantification of the configuration plan indicates that the value 1 indicates that the first configuration plan is adopted, and the number set to -1 indicates that the second configuration plan is adopted.
- the expression value is 1.
- the weight coefficient w of the neuron function of the deep neural network model of the target field is obtained through a large amount of sample data similar to the serial number 1 and the serial number 2 in Table 2.
- the intersection data in Table 2 represents the sample data in the intersection, and the difference data represents the sample data in the difference.
- the sequence numbers 4 and 5 correspond to the difference set data.
- the input vectors (5,7,3) and (8,3,7) are constructed based on the difference set data, and the output y value is -1.
- Table 2 is an example of correcting neuron weight coefficient based on difference set
- the weight parameters of the neuron function of the deep neural network model in the optimized target field are continuously revised, and finally the optimized deep neural network model in the target field is obtained, which realizes automatic recovery and automatic elimination of optical network faults.
- the weight parameters of the corrected and optimized neuron function are stored in each neuron node of the deep neural network model of the target field, as shown in the right part of Figure 6.
- step S300 is based on steps S200 to S220 of the foregoing embodiment, and further optimizes the network fault processing model of the target field based on the difference set.
- step S300 can also be based on steps S110 to S130 of the foregoing embodiment to further optimize the network fault processing model of the target field based on the difference set, which will not be repeated here.
- the embodiment of the present invention also provides a network fault processing method. Based on the foregoing embodiments, the network fault processing method includes:
- S410 acquires alarm data and fault data of the target network, and inputs the network fault processing model after quantitative processing.
- the network fault processing model is obtained by using the aforementioned method for constructing the network fault processing model.
- the output vector of the S420 network fault handling model is delivered to the relevant equipment of the target network.
- the embodiment of the present invention is based on the deep neural network model of the source domain in the optical network, and obtains the network fault handling model of the target domain through cross-domain migration learning.
- the network fault handling model automatically generates configuration data, and Through the management and control platform, the equipment in the target field is issued to complete operations such as equipment restoration, switching, parameter adjustment and rerouting in the target field, so as to realize the self-healing of network failures in the target field.
- the embodiment of the present invention also provides a construction system of a network fault handling model, which is used to implement the construction method of the network fault handling model of the foregoing embodiment.
- the construction system of the network fault handling model includes an acquisition module 100 and a processing module. 200 and building block 300.
- the acquiring module 100 is used to acquire or establish a deep neural network model of the source domain based on the sample set 102 of the source domain in the network.
- the acquisition module 100 includes the acquired source domain sample set 102 and a source domain deep neural network model established based on the source domain sample set 102.
- the acquisition module 100 includes a source domain data collection unit 101, a source domain sample set 102, and a source domain deep neural network model construction unit 103.
- the source domain data collection unit 101 collects sample data and saves it in the source domain sample set 102.
- the deep neural network model of the source domain is constructed by the source domain deep neural network model construction unit 103 based on the source domain sample set 102.
- the processing module 200 is used to establish a sample set 202 of the target domain in the network.
- the sample sets of the target domain and the source domain have an intersection 203, and both include quantitatively processed alarm data, fault data, and configuration data.
- the target field data collection unit 201 in the processing module 200 collects sample data and saves it in the target field sample set 202.
- the processing module 200 is also used to calculate the coincidence rate of the sample sets of the target field and the source field.
- the construction module 300 is used for constructing a network fault processing model of the target domain based on the deep neural network model of the source domain when the coincidence rate of the sample sets of the target domain and the source domain reaches a set threshold.
- the construction module 300 is used to use the deep neural network model of the source domain as the network fault processing model of the target domain; it is also used to extract the first input vector and the corresponding first output vector from the intersection, and retrain the depth of the source domain. Neural network model to obtain the network fault handling model of the target field.
- processing module 200 is also used to obtain the difference set 204 between the sample set in the target field and the sample set in the source field.
- the construction module 300 is used to optimize the network fault handling model of the target field based on the difference set 204.
- construction module 300 is also used for extracting the second input vector and the corresponding second output vector from the difference set 204, and retraining the network fault processing model of the target domain.
- the construction module 300 is also used to extract a third input vector from the difference set 204, and the deep neural network model of the input source domain is used to obtain the third output vector; it is also used to compare the third input vector and the third output vector according to the expert evaluation feedback result. After the output vector is corrected, the network fault handling model in the target field is retrained.
- the construction module 300 is used to modify the weight coefficient of the neuron function of the network fault processing model of the target field based on the difference set 204 to obtain an optimized network fault processing model of the target field.
- the input vector of the deep neural network model of the source domain includes quantized alarm data and fault data
- the output vector is quantized configuration data
- an embodiment of the present invention provides a network fault processing system, which includes an input control module 400, a model processing module 500, and an output control module 600.
- the input control module 400 is used to obtain alarm data and fault data of the target network, and perform quantitative processing.
- the model processing module 500 is used to store the network fault processing model constructed by the aforementioned network fault processing model construction system, and input the quantitatively processed alarm data and fault data into the network fault processing model to obtain the output vector of the network fault processing model.
- the output control module 600 is used to deliver the output vector of the network fault handling model to related devices of the target network.
- the above-mentioned embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
- software it can be implemented in the form of a computer program product in whole or in part.
- the computer program product includes one or more computer instructions.
- the computer program instructions When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.
- the computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices.
- Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
- computer instructions can be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website, computer, server or data center.
- the computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center integrated with one or more available media. Available media can be magnetic media (for example, floppy disks, hard drives, tapes), optical media (for example, Digital Video Disc (DVD)) or semiconductor media (for example, Solid State Disk (SSD)), etc.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention relates to the technical field of communications. Disclosed are a method for constructing a network fault handling model, a fault handling method and a system. The method for constructing a network fault handling model comprises: obtaining or establishing a deep neural network model of a source domain on the basis of a sample set of the source domain in a network; establishing a sample set of a target domain in the network, the sample sets of the target domain and of the source domain having an intersection, and both including quantified alarm data, fault data and configuration data; and if the coincidence rate of the sample sets of the target domain and of the source domain reaches a set threshold, constructing a network fault handling model of the target domain on the basis of the deep neural network model of the source domain. In the present invention, the network fault handling model of the target domain is obtained by means of cross-domain transfer learning on the basis of the deep neural network model of the source domain in an optical network.
Description
本发明涉及通信技术领域,具体是涉及一种网络故障处理模型的构建方法、故障处理方法及***。The invention relates to the field of communication technology, in particular to a method for constructing a network fault processing model, a fault processing method and a system.
光网络设备的当前性能指标越限或者一些潜在性能正在劣化时,会产生一系列的告警数据并上报给网管平台。当光网络设备出现故障时,则会同时产生告警数据和故障数据并上报。目前,运维专家通过分析告警数据和故障数据,定位故障发生位置,制定故障修复策略,然后通过管理平台和控制平台下发相应的配置数据到故障发生位置进行修复,必要时触发保护倒换以保证光网络的正常运行。When the current performance index of the optical network equipment exceeds the limit or some potential performance is deteriorating, a series of alarm data will be generated and reported to the network management platform. When the optical network equipment fails, both alarm data and fault data will be generated and reported. At present, the operation and maintenance experts analyze the alarm data and fault data, locate the fault location, formulate fault repair strategies, and then send the corresponding configuration data to the fault location to repair through the management platform and control platform, and trigger protection switching when necessary to ensure The normal operation of the optical network.
随着光网络规模日益增大,光网络设备不断增多,光网络产生的告警数据和故障数据数量越来越多,网络故障的定位和修复日趋复杂和费力,传统的故障处理模式面监巨大挑战,难以满足实际需要。特别是随着通信业务的飞速发展,通信技术的不断演进和变革,传统紧耦合、刚性网络架构转型为松耦合、灵活的云化网络架构是大势所趋。云化网络底层由光网络设备实现数据转发,中上层通过控制平台、管理平台、编排平台实现资源和业务的管理控制,***运营和维护过程更加复杂,需要实现网络数据融合表示,高效提取数据操作和运算,以解决云化网络出现故障后难以及时恢复的问题。With the ever-increasing scale of optical networks and the increasing number of optical network equipment, the number of alarm data and fault data generated by optical networks is increasing. The location and repair of network faults are becoming more and more complicated and laborious. The traditional fault handling mode faces a huge challenge. , It is difficult to meet actual needs. Especially with the rapid development of communication services and the continuous evolution and change of communication technologies, the transformation of traditional tightly coupled and rigid network architectures into loosely coupled and flexible cloud-based network architectures is the general trend. The bottom layer of the clouded network is forwarded by optical network equipment, and the middle and upper layers realize the management and control of resources and services through the control platform, management platform, and orchestration platform. The system operation and maintenance process is more complicated, and it is necessary to achieve network data integration representation and efficient data extraction operations. And operation to solve the problem of difficulty in timely recovery after a cloud network fails.
采用人工智能技术对网络故障进行分析和修复是应对这些挑战的有效方案。但是,光网络***(特别是云化网络架构)包括具有不 同故障特征的无线网、接入网、承载网和数据中心,一方面,对无线网、接入网、承载网和数据中心分别建立机器学习模型而导致重复学习的问题;另一方面,某些目标领域存在样本数据不完备而难以建立有效的机器学习模型的问题。Using artificial intelligence technology to analyze and repair network faults is an effective solution to these challenges. However, optical network systems (especially cloud network architectures) include wireless networks, access networks, bearer networks, and data centers with different fault characteristics. On the one hand, the wireless network, access network, bearer network, and data center are separately established. The machine learning model leads to the problem of repeated learning; on the other hand, some target fields have incomplete sample data and it is difficult to establish an effective machine learning model.
发明内容Summary of the invention
本发明实施例的目的在于提供一种网络故障处理模型的构建方法、故障处理方法及***,基于光网络中源领域的深度神经网络模型,通过跨领域迁移学习,得到目标领域的网络故障处理模型。The purpose of the embodiments of the present invention is to provide a method for constructing a network fault processing model, a fault processing method and a system, based on the deep neural network model of the source field in the optical network, and through cross-domain migration learning, the network fault processing model of the target field is obtained. .
第一方面,本发明实施例提供一种网络故障处理模型的构建方法,其包括:In the first aspect, an embodiment of the present invention provides a method for constructing a network fault handling model, which includes:
基于所述网络中源领域的样本集,获取或者建立源领域的深度神经网络模型;Acquiring or establishing a deep neural network model of the source domain based on the sample set of the source domain in the network;
建立所述网络中目标领域的样本集,目标领域与源领域的样本集具有交集,且均包括经过量化处理的告警数据、故障数据和配置数据;Establishing a sample set of the target field in the network, the sample set of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data;
当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。When the coincidence rate of the sample sets of the target domain and the source domain reaches the set threshold, the network fault handling model of the target domain is constructed based on the deep neural network model of the source domain.
结合第一方面,在第一种可选的实现方式中,将所述源领域的深度神经网络模型作为所述目标领域的网络故障处理模型;或者,With reference to the first aspect, in the first optional implementation manner, the deep neural network model of the source domain is used as the network fault handling model of the target domain; or,
从所述交集中提取第一输入向量及对应的第一输出向量,重新训练所述源领域的深度神经网络模型,得到所述目标领域的网络故障处理模型。Extracting the first input vector and the corresponding first output vector from the intersection, and retraining the deep neural network model of the source domain to obtain the network fault handling model of the target domain.
结合第一方面,在第二种可选的实现方式中,求取所述目标领域的样本集与源领域的样本集的差集,基于所述差集优化所述目标领域的网络故障处理模型。With reference to the first aspect, in a second optional implementation manner, the difference set between the sample set of the target domain and the sample set of the source domain is obtained, and the network fault handling model of the target domain is optimized based on the difference set .
在第二种可选的实现方式中,从所述差集中提取第二输入向量及 对应的第二输出向量,重新训练所述目标领域的网络故障处理模型。In a second optional implementation manner, a second input vector and a corresponding second output vector are extracted from the difference set, and the network fault handling model in the target domain is retrained.
在第二种可选的实现方式中,从所述差集中提取第三输入向量,输入所述目标领域的网络故障处理模型并得到第三输出向量;In a second optional implementation manner, extract a third input vector from the difference set, input the network fault processing model of the target domain, and obtain a third output vector;
根据专家评估反馈结果对第三输入向量和第三输出向量进行修正后,重新训练所述目标领域的网络故障处理模型。After correcting the third input vector and the third output vector according to the expert evaluation feedback result, the network fault handling model in the target field is retrained.
在第二种可选的实现方式中,基于所述差集对所述目标领域的网络故障处理模型的神经元函数的权重系数进行修正,得到优化的所述目标领域的网络故障处理模型。In a second optional implementation manner, the weight coefficient of the neuron function of the network fault handling model of the target field is corrected based on the difference set to obtain an optimized network fault handling model of the target field.
结合第一方面,在第三种可选的实现方式中,所述源领域的深度神经网络模型的输入向量包括所述经过量化处理的告警数据和故障数据,且输出向量为所述经过量化处理的配置数据。With reference to the first aspect, in a third optional implementation manner, the input vector of the deep neural network model of the source domain includes the quantized alarm data and fault data, and the output vector is the quantized processing Configuration data.
第二方面,本发明实施例提供一种网络故障处理方法,其包括:In a second aspect, an embodiment of the present invention provides a network fault processing method, which includes:
获取目标网络的告警数据和故障数据,经过量化处理后输入网络故障处理模型,所述网络故障处理模型是使用第一方面所述的网络故障处理模型的构建方法得到的;Acquire alarm data and fault data of the target network, and input the network fault processing model after quantitative processing, the network fault processing model being obtained by using the network fault processing model construction method described in the first aspect;
所述网络故障处理模型的输出向量下发到目标网络的相关设备。The output vector of the network fault handling model is delivered to the relevant equipment of the target network.
第三方面,本发明实施例提供一种网络故障处理模型的构建***,其包括:In a third aspect, an embodiment of the present invention provides a construction system for a network fault handling model, which includes:
获取模块,其用于基于所述网络中源领域的样本集,获取或者建立源领域的深度神经网络模型;An acquisition module, which is used to acquire or establish a deep neural network model of the source domain based on the sample set of the source domain in the network;
处理模块,其用于建立所述网络中目标领域的样本集,目标领域与源领域的样本集具有交集,且均包括经过量化处理的告警数据、故障数据和配置数据;以及计算目标领域与源领域的样本集的重合率;A processing module, which is used to establish a sample set of the target field in the network, the sample set of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data; and calculate the target field and the source The coincidence rate of the sample set of the field;
构建模块,其用于当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络 故障处理模型。The construction module is used to construct a network fault handling model of the target domain based on the deep neural network model of the source domain when the coincidence rate of the sample set of the target domain and the source domain reaches a set threshold.
结合第三方面,在第一种可选的实现方式中,所述构建模块用于将所述源领域的深度神经网络模型作为所述目标领域的网络故障处理模型;还用于从所述交集中提取第一输入向量及对应的第一输出向量,重新训练所述源领域的深度神经网络模型,得到所述目标领域的网络故障处理模型。With reference to the third aspect, in a first optional implementation manner, the construction module is used to use the deep neural network model of the source domain as the network fault handling model of the target domain; The first input vector and the corresponding first output vector are concentratedly extracted, and the deep neural network model of the source domain is retrained to obtain the network fault handling model of the target domain.
结合第三方面,在第二种可选的实现方式中,所述处理模块还用于求取所述目标领域的样本集与源领域的样本集的差集;With reference to the third aspect, in a second optional implementation manner, the processing module is further configured to obtain the difference between the sample set of the target field and the sample set of the source field;
所述构建模块还用于基于所述差集优化所述目标领域的网络故障处理模型。The construction module is also used for optimizing the network fault processing model of the target domain based on the difference set.
在第二种可选的实现方式中,所述构建模块用于从所述差集中提取第二输入向量及对应的第二输出向量,重新训练所述目标领域的网络故障处理模型。In a second optional implementation manner, the construction module is configured to extract a second input vector and a corresponding second output vector from the difference set, and retrain the network fault handling model of the target domain.
在第二种可选的实现方式中,所述构建模块用于从所述差集中提取第三输入向量,输入所述目标领域的网络故障处理模型得到第三输出向量;还用于根据专家评估反馈结果对第三输入向量和第三输出向量进行修正后,重新训练所述目标领域的网络故障处理模型。In a second optional implementation manner, the construction module is used to extract a third input vector from the difference set, and input it into the network fault processing model of the target field to obtain a third output vector; After the feedback result corrects the third input vector and the third output vector, the network fault handling model in the target field is retrained.
在第二种可选的实现方式中,所述构建模块用于基于所述差集对所述目标领域的网络故障处理模型的神经元函数的权重系数进行修正,得到优化的所述目标领域的网络故障处理模型。In a second optional implementation manner, the construction module is used to modify the weight coefficients of the neuron function of the network fault handling model of the target field based on the difference set to obtain an optimized value of the target field Network fault handling model.
结合第三方面,在第三种可选的实现方式中,所述源领域的深度神经网络模型的输入向量包括所述经过量化处理的告警数据和故障数据,且输出向量为所述经过量化处理的配置数据。With reference to the third aspect, in a third optional implementation manner, the input vector of the deep neural network model of the source domain includes the quantized alarm data and fault data, and the output vector is the quantized processing Configuration data.
第四方面,本发明实施例提供一种网络故障处理***,其包括:In a fourth aspect, an embodiment of the present invention provides a network fault processing system, which includes:
输入控制模块,其用于获取目标网络的告警数据和故障数据,并 进行量化处理;Input control module, which is used to obtain alarm data and fault data of the target network and perform quantitative processing;
模型处理模块,其用于存储由第三方面所述的网络故障处理模型的构建***构建的网络故障处理模型,并将量化处理后的告警数据和故障数据输入所述网络故障处理模型,得到所述网络故障处理模型的输出向量;The model processing module is used to store the network fault processing model constructed by the network fault processing model construction system described in the third aspect, and input the quantitatively processed alarm data and fault data into the network fault processing model to obtain State the output vector of the network fault handling model;
输出控制模块,其用于将所述网络故障处理模型的输出向量下发到目标网络的相关设备。The output control module is used to deliver the output vector of the network fault handling model to the relevant equipment of the target network.
相对于现有技术,本发明实施例通过一种网络故障处理模型的构建方法,基于网络中源领域的样本集,获取或者建立源领域的深度神经网络模型;建立网络中目标领域的样本集,目标领域与源领域的样本集均包括经过量化处理的告警数据、故障数据和配置数据,且具有交集;当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。基于光网络中源领域的深度神经网络模型,通过跨领域迁移学习,得到目标领域的网络故障处理模型。Compared with the prior art, the embodiment of the present invention uses a method for constructing a network fault handling model to obtain or establish a deep neural network model of the source field based on a sample set of the source field in the network; establish a sample set of the target field in the network, The sample sets of the target field and the source field both include quantified alarm data, fault data, and configuration data, and have an intersection; when the coincidence rate of the sample sets of the target field and the source field reaches the set threshold, the source field-based Deep neural network model to build a network fault handling model in the target field. Based on the deep neural network model of the source domain in the optical network, through cross-domain migration learning, the network fault handling model of the target domain is obtained.
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly describe the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
图1是一种云化网络架构示意图;Figure 1 is a schematic diagram of a cloud network architecture;
图2是本发明实施例网络故障处理模型的构建方法流程图;2 is a flowchart of a method for constructing a network fault handling model according to an embodiment of the present invention;
图3是从数据库获取数据并进行向量化和矩阵化的示意图;Figure 3 is a schematic diagram of obtaining data from a database and performing vectorization and matrixization;
图4是本发明另一实施例网络故障处理模型的构建方法流程图;4 is a flowchart of a method for constructing a network fault handling model according to another embodiment of the present invention;
图5是多层高维空间的一个示例;Figure 5 is an example of a multi-layer high-dimensional space;
图6是目标领域的网络故障处理模型的构建和优化的一个示例;Figure 6 is an example of the construction and optimization of the network fault handling model in the target field;
图7是本发明实施例网络故障处理模型的构建***示意图;FIG. 7 is a schematic diagram of a construction system of a network fault handling model according to an embodiment of the present invention;
图8是本发明实施例网络故障处理***示意图。Fig. 8 is a schematic diagram of a network fault handling system according to an embodiment of the present invention.
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the protection scope of the present invention.
本发明实施例涉及的网络既可以是(Optical Transport Network,OTN)、分组传送网(Packet Transport Network,PTN)和分组光传送网络(Packet Optical Transport Network,POTN)等传统的光传送网,还可以是云化网络。The network involved in the embodiment of the present invention may be a traditional optical transport network such as (Optical Transport Network, OTN), Packet Transport Network (PTN), and Packet Optical Transport Network (Packet Optical Transport Network, POTN), or It is a cloud network.
作为一个示例,图1是一种云化网络架构示意图,图1左下部分是云化网络基站,包括有源天线单元(Active Antenna Unit,AAU)、集中单元(Centralized Unit,CU)和分布式单元(Distributed Unit,DU),其中,CU支持非实时无线高层协议以及部分核心网下沉功能和边缘应用功能,DU支持物理层功能和实时功能。图1下部是云化网络接入环、汇聚环和核心环,这些环形网络中的网络设备的告警数据、故障数据和配置数据通过网络管理平台或者控制器平台分别上报至图1上部的边缘数据中心、区域数据中心和核心数据中心,基站和边缘应用的告警数据、故障数据和配置数据通过本地网上报至边缘数据中心。5G核心网络的核心网功能分为用户面(User Plane,UP)功能与控制面(Control Plane,CP)功能。这些数据中心一方面承担着云化网络的管理、编排和控制等功能,另一方面部署云化网络的智能 化平台,基于海量网络数据和强大的计算能力,构建云化网络运维管理知识库,担任云化网络的大脑。As an example, Figure 1 is a schematic diagram of a cloudized network architecture. The bottom left part of Figure 1 is a cloudized network base station, including Active Antenna Unit (AAU), Centralized Unit (CU), and Distributed Unit (Distributed Unit, DU), where CU supports non-real-time wireless high-level protocols and some core network sink functions and edge application functions, and DU supports physical layer functions and real-time functions. The lower part of Figure 1 is the cloudized network access ring, aggregation ring and core ring. The alarm data, fault data and configuration data of the network devices in these ring networks are reported to the edge data in the upper part of Figure 1 through the network management platform or the controller platform. Center, regional data center and core data center, base station and edge application alarm data, fault data and configuration data are reported to edge data center through local network. The core network functions of the 5G core network are divided into user plane (UP) functions and control plane (CP) functions. On the one hand, these data centers are responsible for the management, orchestration, and control of cloud-based networks. On the other hand, they deploy intelligent platforms for cloud-based networks, and build a cloud-based network operation and maintenance management knowledge base based on massive network data and powerful computing capabilities. , As the brain of the cloud network.
因为海量的光网络告警数据、故障数据和配置数据中包含大量重复冗余、不完备和不一致的数据,数据中心首先对数据进行清洗,去除重复冗余、低质量数据,得到高质量的告警数据集、故障数据集和配置数据集,并分别保存在数据库中。Because the massive amount of optical network alarm data, fault data, and configuration data contains a large amount of redundant, incomplete and inconsistent data, the data center first cleans the data to remove redundant, low-quality data and obtain high-quality alarm data Sets, fault data sets, and configuration data sets are stored in the database.
在本发明实施例中,以图1为例,源领域可以定义为接入网,目标领域定义为汇聚网,或者源领域定义为核心网,目标领域定义为数据中心网络,不作限定。云化网络不同网络的设备可能有各自的专业网管或专用控制平台。在其他的实施例中,源领域和目标领域还可以分别是传统光网络(OTN、PTN和POTN)中的接入网、汇聚网和核心网。In the embodiment of the present invention, taking FIG. 1 as an example, the source domain may be defined as an access network, the target domain may be defined as an aggregation network, or the source domain may be defined as a core network, and the target domain may be defined as a data center network, which is not limited. Devices on different networks of clouded networks may have their own professional network management or dedicated control platforms. In other embodiments, the source domain and the target domain may also be an access network, an aggregation network, and a core network in a traditional optical network (OTN, PTN, and POTN), respectively.
网络设备将告警数据和相关的故障数据上报网络管理平台,由网络管理平台提交至数据中心。网络设备产生的告警包括根源告警和衍生告警,根源告警和衍生告警之间相关联。网络设备出现故障时,同时产生告警数据和故障数据并上报,并需要通过下发的配置数据对故障进行修复。The network equipment reports the alarm data and related fault data to the network management platform, and the network management platform submits it to the data center. Alarms generated by network equipment include root cause alarms and derivative alarms, and there is a correlation between root cause alarms and derivative alarms. When a network device fails, alarm data and fault data are generated and reported at the same time, and the fault needs to be repaired through the issued configuration data.
本发明实施例基于网络中源领域的深度神经网络模型,通过跨领域迁移学***台下发目标领域的设备,完成目标领域设备恢复、导换、调参和重路由等操作,从而实现目标领域的网络故障自愈。The embodiment of the present invention is based on the deep neural network model of the source domain in the network, and obtains the network fault processing model of the target domain through cross-domain migration learning. Therefore, when an alarm or failure occurs in the target field, the network fault handling model in the target field automatically generates configuration data, and distributes the equipment in the target field through the management control platform to complete the equipment recovery, switching, parameter adjustment and rerouting of the target field, etc. Operate to achieve self-healing of network failures in the target area.
本发明实施例解决了网络中不同领域建立网络故障处理模型时重复学习,以及某些目标领域存在样本数据不完备而难以建立有效的机器学习模型的问题,而且有利于对网络中不同领域进行统一管理。The embodiment of the present invention solves the problem of repeated learning when establishing network fault handling models in different fields in the network, and the problem of incomplete sample data in some target fields, which makes it difficult to establish an effective machine learning model, and is conducive to unifying different fields in the network management.
图2所示为本发明实施例网络故障处理模型的构建方法流程图,网络包括源领域和目标领域,网络故障处理模型的构建方法包括:Figure 2 shows a flowchart of a method for constructing a network fault handling model according to an embodiment of the present invention. The network includes a source field and a target field. The method for constructing a network fault handling model includes:
S110获取源领域的样本集及其深度神经网络模型。S110 obtains the sample set of the source domain and its deep neural network model.
S120建立目标领域的样本集,目标领域与源领域的样本集具有交集,且均包括经过量化处理的告警数据、故障数据和配置数据。S120 establishes a sample set of the target field. The sample sets of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data.
S130当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。S130: When the coincidence rate of the sample sets of the target domain and the source domain reaches the set threshold, construct a network fault handling model of the target domain based on the deep neural network model of the source domain.
在步骤S110中,源领域的深度神经网络模型是基于源领域的样本集预先创建的。常见的深度神经网络模型包括栈式自编码器(Stacked Auto-Encoder)、卷积神经网络(Convolutional Neural Network,CNN)和深度置信网络(Deep Belief Network)等。In step S110, the deep neural network model of the source domain is created in advance based on the sample set of the source domain. Common deep neural network models include Stacked Auto-Encoder, Convolutional Neural Network (CNN), Deep Belief Network, etc.
源领域的样本集包括经过量化处理的告警数据、故障数据和配置数据,参见步骤S120中的具体说明。The sample set in the source domain includes quantitatively processed alarm data, fault data, and configuration data, see the specific description in step S120.
源领域的深度神经网络模型的输入和输出样本数据通常采用向量形式,即样本集包括根据源领域的告警数据、故障数据和配置数据,分别得到的告警数据向量组、故障数据向量组以及配置数据向量组。The input and output sample data of the deep neural network model in the source field usually takes the form of vectors, that is, the sample set includes the alarm data vector group, the fault data vector group and the configuration data respectively obtained according to the alarm data, fault data and configuration data of the source field Vector set.
作为一个示例,源领域的深度神经网络模型的输入向量包括经过量化处理的告警数据和故障数据,且输出向量为经过量化处理的配置数据。As an example, the input vector of the deep neural network model of the source domain includes quantized alarm data and fault data, and the output vector is quantized configuration data.
采用人工智能深度学***台,实现光网源领域故障的自动修复。Using the artificial intelligence deep learning method, the quantified alarm data and fault data are used as input, and the quantized configuration data is used as the output. A deep neural network model is generated and trained. Through large-scale high-quality sample data training, the depth The neural network model learns the fault intelligent self-healing knowledge in the source field, and the relevant knowledge is stored in a series of neurons in the deep neural network in an abstract form. Through the deep neural network model in the source domain, the association rules between the optical network-derived alarms and the root-cause alarms are excavated, and the precise relationship between the root-cause alarms and the fault location is generated. The network configuration plan can be given according to the alarm and fault information, and the network management and The controller platform realizes the automatic repair of faults in the optical network source field.
在步骤S120中,从目标领域的数据库中获取多个时间点的告警数据、故障数据和配置数据,经过量化处理后得到目标领域的样本集。其中,为了获得目标领域与源领域的样本集的交集,源领域和目标领域的告警数据、故障数据和配置数据的字段定义相同,但是排序并不要求必须相同。In step S120, the alarm data, fault data and configuration data at multiple time points are obtained from the database of the target field, and the sample set of the target field is obtained after quantization processing. Among them, in order to obtain the intersection of the sample sets of the target field and the source field, the field definitions of the alarm data, fault data, and configuration data of the source field and the target field are the same, but the ordering is not required to be the same.
基于告警数据、故障数据和配置数据在产生的时间上具有相关性,可以从数据库中获取设定的时间段内目标领域的所有告警数据、故障数据和配置数据,也可以按天、周或者月等周期性地从数据库中获取目标领域的所有告警数据、故障数据和配置数据。设定的时间段或者周期内包括多个时间点的告警数据,多个时间点的故障数据以及多个时间点的配置数据。Based on the correlation between the alarm data, fault data and configuration data in the generation time, all the alarm data, fault data and configuration data of the target field in the set time period can be obtained from the database, or by day, week or month Periodically obtain all alarm data, fault data and configuration data of the target field from the database. The set time period or cycle includes alarm data at multiple time points, fault data at multiple time points, and configuration data at multiple time points.
告警数据、故障数据和配置数据不仅是异构数据,而且这些数据包括各种类型的字段,而且不同的字段有不同的量纲。因此,量化处理包括对不同量纲的异构数据的向量化表示,包括:Alarm data, fault data, and configuration data are not only heterogeneous data, but these data include various types of fields, and different fields have different dimensions. Therefore, the quantization process includes the vectorized representation of heterogeneous data of different dimensions, including:
S121每条告警数据、故障数据或者配置数据都被转换为一个基础向量V
b,基础向量V
b的每个元素为每条告警数据、故障数据或者配置数据中一个字段的数值。
In S121, each piece of alarm data, fault data or configuration data is converted into a basic vector V b , and each element of the basic vector V b is the value of a field in each piece of alarm data, fault data or configuration data.
例如,获取的所有告警数据所构成的样本集有M
a条告警数据,其中,在一个时间点上产生的告警数据可以是一条或者多条,每条告警数据有N
a个字段。
For example, the sample data set acquired all alarms are constituted M a in a trap data, wherein the alerting data generated at a time point may be one or more pieces, each alarm data field has N a.
作为一个示例,图3中所示的一条告警数据包括八个字段,分别 是:告警数据的序列号Seq.No.、地址Addr.、线路号Line、告警类型AlarmType、告警开始时间BeginTime、告警结束时间EndTime、板类型BoardType和网元类型NetType,其中,告警开始时间BeginTime和告警结束时间EndTime精确到秒,地址Addr.和告警类型AlarmType为字符号,网元类型NetType为整型值。As an example, a piece of alarm data shown in Figure 3 includes eight fields, namely: the sequence number of the alarm data Seq.No., address Addr., line number Line, alarm type AlarmType, alarm start time BeginTime, alarm end Time EndTime, board type BoardType, and network element type NetType, where the alarm start time BeginTime and alarm end time EndTime are accurate to seconds, the address Addr. and alarm type AlarmType are character numbers, and the network element type NetType is an integer value.
将图3所示告警数据的所有字段的值转换为实数,从而表示为向量的元素。在告警数据的向量化过程中,这些字段的整型值作为元素值表示在向量中。可以将所有告警开始时间BeginTime和告警结束时间EndTime中的最小值对应为数值1,其他时间与最小时间相差的秒数加到数值1上,分别得到告警开始时间BeginTime和告警结束时间EndTime的对应值。例如,告警开始时间BeginTime比最小时间多10秒,则告警开始时间BeginTime对应数值11,将这两个字段按字典序进行排列,然后从1进行编号,将字符串转换为数值后作为向量的元素。The values of all fields of the alarm data shown in Fig. 3 are converted into real numbers, and thus expressed as elements of a vector. In the vectorization process of alarm data, the integer values of these fields are represented in the vector as element values. The minimum value of all alarm start time BeginTime and alarm end time EndTime can be corresponding to the value 1, and the number of seconds between other times and the minimum time can be added to the value 1, and the corresponding values of the alarm start time BeginTime and the alarm end time EndTime can be obtained respectively . For example, if the alarm start time BeginTime is 10 seconds longer than the minimum time, the alarm start time BeginTime corresponds to the value 11. The two fields are arranged in lexicographic order, and then numbered from 1, and the string is converted to a value as an element of the vector .
S122对基础向量V
b进行量纲转换,转换得到的向量V为基础向量V
b与量纲扩展向量V
s的hadamard积,即
量纲扩展向量V
s的元素为基础向量V
b的相应元素的扩大或者缩小倍数,例如将带宽单位兆M扩大为千兆G,则量纲扩展向量V
s的元素为1024。
S122 performs dimension conversion on the basis vector V b , and the converted vector V is the hadamard product of the basis vector V b and the dimension expansion vector V s , namely The element of the dimension expansion vector V s is the expansion or reduction multiple of the corresponding element of the basic vector V b . For example, if the bandwidth unit M is expanded to giga G, the element of the dimension expansion vector V s is 1024.
根据机器学习模型训练的要求,可以将基础向量与量纲扩展向量的对应元素相乘,生成适合训练要求的样本数据。同理,图3左下部分的配置数据和故障数据也转换为相应的向量,配置数据包括Num_CPUs:4,即CPU的内核数量,作为示例,图3下部的向量组显示了两个向量,分别由告警数据和配置数据转换得到。According to the training requirements of the machine learning model, the basic vector can be multiplied by the corresponding elements of the dimension expansion vector to generate sample data suitable for the training requirements. In the same way, the configuration data and fault data in the lower left part of Fig. 3 are also converted into corresponding vectors. The configuration data includes Num_CPUs: 4, which is the number of CPU cores. As an example, the vector group in the lower part of Fig. 3 shows two vectors. The alarm data and configuration data are converted.
对于光网络中保存在半结构化XML文档中的故障数据和配置数据,同样可以利用上述方法构建数据基础向量和量纲扩展向量,XML 中键值对(Key/Value)的个数对应向量的维度,向量元素的值对应XML文档中的Value值。For the fault data and configuration data stored in the semi-structured XML document in the optical network, the above method can also be used to construct the data basis vector and the dimension expansion vector. The number of key/value pairs in XML corresponds to the number of the vector Dimension, the value of the vector element corresponds to the Value value in the XML document.
为目标领域构建三对向量组,分别为告警数据基础向量组与量纲扩展向量组、故障数据基础向量组与量纲扩展向量组、以及配置数据基础向量组与量纲扩展向量组,得到的告警数据向量组包括由M
a条告警数据转换得到的M
a个告警数据向量,每个告警数据向量具有N
a个元素;故障数据向量组包括由M
f条故障数据转换得到的M
f个故障数据向量,每个故障数据向量具有N
f个元素;配置数据向量组包括由M
c条配置数据转换得到的M
c个配置数据向量,每个配置数据向量具有N
c个元素。
Construct three pairs of vector groups for the target field, which are the basic vector group of alarm data and the expanded vector group of dimensions, the basic vector group of fault data and the expanded vector group of dimensions, and the basic vector group of configuration data and the expanded vector group of dimensions. including alarm data vectors obtained by the data converting M a M a in a trap number alarm data vectors, each data vector having N a warning elements; vector data group includes fault fault M f M f obtained from the data conversion section fault data vectors, each data vector having N f fault elements; vector set of configuration data including configuration data obtained by conversion of configuration data vector M c M c from the bar, each configuration data vector with N c elements.
进一步的,还可以对告警数据向量组、故障数据向量组和配置数据向量组进行矩阵化表示,例如,告警数据向量组以行向量的方式存入一个二维的空矩阵中,形成告警矩阵,例如图3右下部分的二维矩阵,假如有M
a=7000条告警数据,则形成7000行8列的告警矩阵。同样的,可以构建出故障矩阵和配置矩阵。
Further, the alarm data vector group, the fault data vector group, and the configuration data vector group can also be expressed in matrix. For example, the alarm data vector group is stored in a two-dimensional empty matrix in the form of row vectors to form an alarm matrix. for example, two-dimensional matrix of the lower right portion of FIG. 3, M a = 7000 if there is data in a trap, trap matrix is formed 7000 rows and 8 columns. Similarly, fault matrix and configuration matrix can be constructed.
S123求取目标领域与源领域的样本集的交集。S123 finds the intersection of the sample set of the target field and the source field.
具体的,根据目标领域与源领域的告警数据向量组、故障数据向量组以及配置数据向量组中向量元素,求取目标领域与源领域的样本集的交集。Specifically, according to the vector elements in the alarm data vector group, the fault data vector group and the configuration data vector group of the target domain and the source domain, the intersection of the sample sets of the target domain and the source domain is obtained.
在步骤S130中,当源领域的样本集与目标领域的样本集的重合率达到设定的阈值时,构建目标领域的网络故障处理模型可以采用不同的实施方式,例如可以采用以下的实施方式之一:In step S130, when the coincidence rate of the sample set of the source field and the sample set of the target field reaches the set threshold, the network fault handling model of the target field can be constructed in different implementation manners, for example, one of the following implementation manners can be adopted One:
实施方式一:将源领域的深度神经网络模型作为目标领域的网络故障处理模型。Implementation mode 1: The deep neural network model in the source domain is used as the network fault processing model in the target domain.
实施方式二:从交集中提取第一输入向量及对应的第一输出向 量,重新训练源领域的深度神经网络模型,得到目标领域的网络故障处理模型,目标领域的网络故障处理模型是与源领域的深度神经网络模型相似的深度神经网络模型。Embodiment 2: Extract the first input vector and the corresponding first output vector from the intersection, retrain the deep neural network model of the source domain, and obtain the network fault handling model of the target domain. The network fault handling model of the target domain is the same as that of the source domain. The deep neural network model is similar to the deep neural network model.
由源领域的深度神经网络模型得到源领域故障自愈的知识库以后,求取源领域和目标领域的样本数据的交集,由于交集的故障自愈知识已经包含在源领域故障自愈的知识库中,因此,基于源领域和目标领域的样本数据的交集,将源领域故障自愈的知识库迁移到目标领域中,实现跨领域迁移学习。迁移学习过程中,如果源领域和目标领域的样本数据的交集比较大(即重合率较高),迁移学习效果就会比较好。After the source domain's deep neural network model obtains the source domain fault self-healing knowledge base, the intersection of the sample data of the source domain and the target domain is obtained. Because the intersection fault self-healing knowledge is already included in the source domain fault self-healing knowledge base Therefore, based on the intersection of the sample data of the source domain and the target domain, the knowledge base for self-healing faults in the source domain is migrated to the target domain to realize cross-domain migration learning. In the process of transfer learning, if the intersection of the sample data in the source field and the target field is relatively large (that is, the coincidence rate is higher), the transfer learning effect will be better.
在实际应用中,阈值的大小可根据具体场景进行调整,阈值是一个百分比数值,例如,源领域与目标领域的数据交集的重合率在0%至100%之间。例如,重合率60%表示源领域与目标领域的样本数据有60%相同,40%不相同。In practical applications, the size of the threshold can be adjusted according to specific scenarios. The threshold is a percentage value. For example, the overlap rate of the data intersection of the source domain and the target domain is between 0% and 100%. For example, a coincidence rate of 60% means that the sample data of the source field and the target field are 60% the same, and 40% are different.
如果阈值较小,则源领域故障自愈的知识库的迁移过程较快,而后续权重参数的修正和优化过程较长。反之,如果阈值较大,则源领域故障自愈的知识库的迁移过程较慢,但后续权重参数的修正和优化过程较短。If the threshold is small, the migration process of the knowledge base of the source domain fault self-healing is faster, and the subsequent correction and optimization process of the weight parameter is longer. Conversely, if the threshold is larger, the migration process of the knowledge base for self-healing failures in the source domain is slower, but the subsequent correction and optimization process of the weight parameters is shorter.
如果重合率低于设定的阈值,则需要在目标领域的样本集中增加新的数据,也可以再次挑选一批数据样本,分别补充源领域和目标领域的交集数据,直至重合率超过设定的阈值。If the coincidence rate is lower than the set threshold, you need to add new data to the sample set of the target field, or you can select a batch of data samples again to supplement the intersection data of the source field and the target field, until the coincidence rate exceeds the set Threshold.
在本实施例中,步骤S110和S120依序执行,而在本发明另一实施例中,步骤S110和S120也可以采用其他执行方式,例如,分别获取源领域和目标领域的告警数据、故障数据和配置数据,经过量化处理后分别建立源领域和目标领域的样本集,然后构建源领域的深度神 经网络模型。In this embodiment, steps S110 and S120 are executed in sequence, and in another embodiment of the present invention, steps S110 and S120 can also be executed in other ways, for example, obtaining alarm data and fault data in the source domain and target domain respectively After quantitative processing, the sample sets of the source field and the target field are established respectively, and then the deep neural network model of the source field is constructed.
图4所示为本发明另一实施例网络故障处理模型的构建方法流程图,网络故障处理模型的构建方法包括:Fig. 4 is a flowchart of a method for constructing a network fault handling model according to another embodiment of the present invention. The method for constructing a network fault handling model includes:
S200数据采集与预处理。其具体包括:S200 data collection and preprocessing. It specifically includes:
S201源领域的数据采集与预处理。S201 data collection and preprocessing in the source field.
S202目标领域的数据采集与预处理。S202 Data collection and preprocessing in the target field.
源领域和目标领域的数据采集与预处理过程基本相同。The data collection and preprocessing process of the source field and target field are basically the same.
光网络的告警数据、故障数据和配置数据由网络管理平台或者控制器平台上传至三类数据中心。因为海量的光网络的告警数据、故障数据和配置数据中包含大量重复冗余、不完备、不一致的数据,三类数据中心会首先对数据进行清洗,去除重复冗余低质量数据,求得高质量的告警、故障、配置数据集,并分别保存在源领域数据库和目标领域数据库中。The alarm data, fault data and configuration data of the optical network are uploaded to the three types of data centers by the network management platform or the controller platform. Because the alarm data, fault data, and configuration data of the massive optical network contain a large amount of redundant, incomplete, and inconsistent data, the three types of data centers will first clean the data, remove the redundant, low-quality data, and get the highest Quality alarms, faults, and configuration data sets are stored in the source domain database and target domain database respectively.
S210构建源领域的深度神经网络模型。S210 constructs a deep neural network model of the source domain.
源领域的深度神经网络模型的构建方法不作限定,例如,常见的深度神经网络模型包括栈式自编码器(Stacked Auto-Encoder)、卷积神经网络(Convolutional Neural Network,CNN)和深度置信网络(Deep Belief Network)等。The method of constructing deep neural network models in the source domain is not limited. For example, common deep neural network models include Stacked Auto-Encoder, Convolutional Neural Network (CNN), and Deep Belief Network ( Deep Belief Network) and so on.
S220当源领域的样本集与目标领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。S220 When the coincidence rate of the sample set of the source domain and the sample set of the target domain reaches the set threshold, construct a network fault handling model of the target domain based on the deep neural network model of the source domain.
步骤S220具体包括:Step S220 specifically includes:
S221源领域的样本集和目标领域的样本集的统一表示。S221 Unified representation of the sample set of the source field and the sample set of the target field.
具体的,构建多层高维空间,实现源领域和目标领域的告警数据、故障数据和配置数据的统一表示。Specifically, a multi-layer high-dimensional space is constructed to realize the unified representation of alarm data, fault data, and configuration data in the source and target fields.
依次采用不同量纲异构数据的向量化和矩阵化表示方法,将源领域和目标领域的告警数据、故障数据和配置数据分别转换为一维向量,然后分别表示成对应的二维矩阵。其具体包括:一维向量的构建过程,以及二维矩阵的构建过程。The vectorization and matrix representation methods of heterogeneous data of different dimensions are successively used to convert the alarm data, fault data and configuration data of the source and target fields into one-dimensional vectors, and then respectively express them into corresponding two-dimensional matrices. It specifically includes: the construction process of one-dimensional vector and the construction process of two-dimensional matrix.
具体的,根据源领域的告警数据、故障数据和配置数据分别构建二维的告警矩阵、故障矩阵和配置矩阵,根据目标领域的告警数据、故障数据和配置数据分别构建二维的告警矩阵、故障矩阵和配置矩阵,一维向量和二维矩阵的构建方法与前述实施例相似,此处不再赘述。Specifically, a two-dimensional alarm matrix, a fault matrix, and a configuration matrix are constructed according to the alarm data, fault data, and configuration data of the source field, and a two-dimensional alarm matrix, fault, and configuration data are constructed according to the alarm data, fault data, and configuration data of the target field. The construction methods of the matrix and the configuration matrix, the one-dimensional vector and the two-dimensional matrix are similar to the foregoing embodiments, and will not be repeated here.
作为一个示例,假如源领域和目标领域的告警数据、故障数据和配置数据在矩阵化表示后得到的二维矩阵的行数和列数可能不同,如下表1:As an example, if the alarm data, fault data, and configuration data of the source field and the target field are expressed in a matrix, the number of rows and columns of the two-dimensional matrix may be different, as shown in Table 1:
表1 源领域和目标领域的二维矩阵的行数和列数示例Table 1 Examples of the number of rows and columns of the two-dimensional matrix of the source and target fields
矩阵类型Matrix type | 告警矩阵的行列数Number of rows and columns of the alarm matrix | 故障矩阵的行列数Number of rows and columns of the fault matrix | 配置矩阵的行列数Configure the number of rows and columns of the matrix |
源领域Source field | 5000×125000×12 | 7000×187000×18 | 3000×323000×32 |
目标领域Target field | 3000×83000×8 | 5000×125000×12 | 2000×352000×35 |
求取所有告警矩阵、故障矩阵和配置矩阵的最大行数和最大列数,将最大行数和最大列数作为多层高维空间的每层二维矩阵的行数和列数。以表一为例,则多层高维空间的每层二维矩阵的行数和列数分别为7000和35。其中,行数7000是指六个矩阵中最大的行数是源领域故障矩阵的行数,列数35是指六个矩阵中最大的列数是目标领域配置矩阵的列数。Obtain the maximum number of rows and the maximum number of columns of all alarm matrix, fault matrix, and configuration matrix, and use the maximum number of rows and the maximum number of columns as the number of rows and columns of each layer of the two-dimensional matrix in the multi-layer high-dimensional space. Taking Table 1 as an example, the number of rows and columns of each layer of a two-dimensional matrix in a multi-layer high-dimensional space are 7000 and 35, respectively. Among them, the number of rows 7000 means that the largest number of rows in the six matrices is the number of rows of the source field fault matrix, and the number of columns 35 means that the largest number of columns in the six matrices is the number of columns of the target field configuration matrix.
求得最大行数7000和最大列数35后,基于上述表一中的六个矩阵构建一个6层的高维空间表示模型,生成6个7000行和35列的空矩阵,并将这6个矩阵中的数据复制至新生成的空矩阵中,没有存储 数据的矩阵元素用零元素填充。After obtaining the maximum number of rows of 7000 and the maximum number of columns of 35, a 6-layer high-dimensional space representation model is constructed based on the six matrices in Table 1 above, and 6 empty matrices with 7000 rows and 35 columns are generated, and these 6 The data in the matrix is copied to the newly generated empty matrix, and the matrix elements without stored data are filled with zero elements.
具体的,为源领域和目标领域构建的多层高维空间如图5所示,六层的多层高维空间D=R(K
1,K
2,K
3),第一层至第三层为源领域的告警数据层、故障数据层和配置数据层,分别对应源领域的告警矩阵、故障矩阵和配置矩阵,第四层至第六层为目标领域的告警数据层、故障数据层和配置数据层,分别对应目标领域的告警矩阵、故障矩阵和配置矩阵。其中,源领域的三层高维空间还可以表示为D
s=R(I
1,I
2,I
3),目标领域的三层高维空间还可以表示为D
t=R(J
1,J
2,J
3)。
Specifically, the multi-layer high-dimensional space constructed for the source field and the target field is shown in Figure 5. The six-layer multi-layer high-dimensional space D=R(K 1 , K 2 , K 3 ), the first to the third The layers are the alarm data layer, fault data layer and configuration data layer of the source field, corresponding to the alarm matrix, fault matrix and configuration matrix of the source field respectively. The fourth to sixth layers are the alarm data layer, fault data layer and The configuration data layer corresponds to the alarm matrix, fault matrix and configuration matrix of the target field. Among them, the three-layer high-dimensional space in the source field can also be expressed as D s = R(I 1 , I 2 , I 3 ), and the three-layer high-dimensional space in the target field can also be expressed as D t =R(J 1 , J 2 , J 3 ).
采用以上实施例中的方法,还可以为多个领域构建一个多层高维空间,例如接入网、汇聚网、核心网和数据中心网络,不作限定。Using the method in the above embodiment, it is also possible to construct a multi-layer high-dimensional space for multiple fields, such as an access network, a convergence network, a core network, and a data center network, which is not limited.
通过本发明实施例,对不同量纲的异构数据的向量化和矩阵化表示方法,能够将不同量纲的结构化、半结构化的光网络数据转换为向量和矩阵,因为有大量零元素填充,多层高维空间是稀疏矩阵,在保存过程中,可以采用经典的稀疏矩阵存储方法进行数据保存,以便节省存储空间。同时,构建多层高维空间不但实现源领域和目的领域的样本数据的统一表示,而且能够实现异厂商的跨域样本数据的互通和共享,为后续机器学习扫除信息孤岛障碍。Through the embodiment of the present invention, the vectorization and matrix representation methods for heterogeneous data of different dimensions can convert structured and semi-structured optical network data of different dimensions into vectors and matrices, because there are a large number of zero elements Filling, multi-layer high-dimensional space is a sparse matrix. In the process of saving, the classic sparse matrix storage method can be used to save data to save storage space. At the same time, constructing a multi-layer high-dimensional space not only realizes the unified representation of sample data in the source domain and the target domain, but also realizes the intercommunication and sharing of cross-domain sample data from different vendors, removing information island barriers for subsequent machine learning.
源领域的样本集可以是源领域的三层高维空间D
s=R(I
1,I
2,I
3),也可以是D
s=R(I
1,I
2,I
3)的一个子空间。同样地,目标领域的样本集可以是目标领域的三层高维空间D
t=R(J
1,J
2,J
3),也可以是D
t=R(J
1,J
2,J
3)的一个子空间。
The sample set of the source domain can be a three-layer high-dimensional space D s = R(I 1 , I 2 , I 3 ) in the source domain, or a sub-sub of D s = R(I 1 , I 2 , I 3 ) space. Similarly, the sample set of the target field can be the three-layer high-dimensional space D t =R(J 1 , J 2 , J 3 ) of the target field, or D t =R(J 1 , J 2 , J 3 ) Of a subspace.
子空间包括告警数据层、故障数据层和配置数据层中的至少一个子矩阵。子矩阵可以是多层高维空间的一层中的一个子矩阵;子矩阵也可以是多层高维空间的二层或以上,其中,子矩阵的每层为多层高维空间的一层的一个子矩阵。The subspace includes at least one submatrix of an alarm data layer, a fault data layer, and a configuration data layer. The sub-matrix can be a sub-matrix in one layer of the multi-layer high-dimensional space; the sub-matrix can also be two or more layers of the multi-layer high-dimensional space, where each layer of the sub-matrix is a layer of the multi-layer high-dimensional space A sub-matrix of.
作为一个示例,图6中的矩阵S和矩阵T分别表示迁移学习过程中源领域的样本集和目标领域的样本集,这两个矩阵都是3行3列的矩阵。As an example, the matrix S and the matrix T in FIG. 6 respectively represent the sample set of the source field and the sample set of the target field in the transfer learning process, and these two matrices are both matrices with 3 rows and 3 columns.
S222求取目标领域与源领域的样本集的交集。S222 finds the intersection of the sample set of the target domain and the source domain.
具体的,该交集也是多层高维空间的子空间,子空间包括告警数据层、故障数据层和配置数据层中的至少一个子矩阵。Specifically, the intersection is also a subspace of a multi-layer high-dimensional space, and the subspace includes at least one submatrix of an alarm data layer, a fault data layer, and a configuration data layer.
子矩阵可以是多层高维空间的一层中的一个子矩阵;子矩阵也可以是多层高维空间的二层或以上,其中,子矩阵的每层为多层高维空间的一层的一个子矩阵。The sub-matrix can be a sub-matrix in one layer of the multi-layer high-dimensional space; the sub-matrix can also be two or more layers of the multi-layer high-dimensional space, where each layer of the sub-matrix is a layer of the multi-layer high-dimensional space A sub-matrix of.
还是以图6中的矩阵S和矩阵T为例,矩阵S和矩阵T分别表示迁移学习过程中源领域的样本集和目标领域的样本集,这两个矩阵都是3行3列的矩阵。求取矩阵S和T的数据交集,得到交集矩阵I,得到一个2行3列的矩阵,即图6中矩阵S的第一个行向量和矩阵T的第一个行向量相等,即S
11=T
11,S
12=T
12,S
13=T
13,并且矩阵S的第二个行向量和矩阵T的第二个行向量相等,即S
21=T
21,S
22=T
22,S
23=T
23,则说明矩阵S和矩阵T的前两个行向量相等,将这两个相等的行向量取出来得到交集I。
Take the matrix S and the matrix T in Fig. 6 as examples. The matrix S and the matrix T respectively represent the sample set of the source field and the sample set of the target field in the transfer learning process. Both matrices are three-row and three-column matrices. Find the data intersection of the matrix S and T, get the intersection matrix I, get a matrix with 2 rows and 3 columns, that is, the first row vector of the matrix S in Figure 6 is equal to the first row vector of the matrix T, that is, S 11 = T 11 , S 12 = T 12 , S 13 = T 13 , and the second row vector of the matrix S is equal to the second row vector of the matrix T, that is, S 21 = T 21 , S 22 = T 22 , S 23 = T 23 , it means that the first two row vectors of the matrix S and the matrix T are equal, and the two equal row vectors are taken out to obtain the intersection I.
S223构建目标领域的深度神经网络模型。S223 builds a deep neural network model of the target field.
步骤S223与前述实施例中步骤S130基本相同。Step S223 is basically the same as step S130 in the foregoing embodiment.
具体的,源领域的样本集与目标领域的样本集的重合部分即交集,可以直接将源领域的深度神经网络模型作为目标领域的网络故障处理模型,或者从交集中提取第一输入向量及对应的第一输出向量,重新训练源领域的深度神经网络模型,得到目标领域的网络故障处理模型,从而将源领域的故障处理知识库迁移至目标领域的故障处理知识库。Specifically, the overlap between the sample set of the source field and the sample set of the target field is the intersection. The deep neural network model of the source field can be directly used as the network fault processing model of the target field, or the first input vector and the corresponding can be extracted from the intersection. Retrain the deep neural network model of the source domain to obtain the network fault handling model of the target domain, thereby migrating the fault handling knowledge base of the source domain to the fault handling knowledge base of the target domain.
还是以图6示出的例子进行说明,图6中矩阵S和T为9个元素的矩阵,交集矩阵I是6个元素的矩阵,如果设定的阈值为60%,交集数据占比超过设定的阈值60%,可以直接将源领域的深度神经网络模型作为目标领域的网络故障处理模型,或者,从交集中提取第一输入向量及对应的第一输出向量,重新训练源领域的深度神经网络模型,得到目标领域的网络故障处理模型。Still take the example shown in Figure 6 for explanation. In Figure 6, the matrices S and T are 9-element matrices, and the intersection matrix I is a 6-element matrix. If the threshold is set to 60%, the proportion of the intersection data exceeds the set Set a threshold of 60%, you can directly use the deep neural network model in the source field as the network fault handling model in the target field, or extract the first input vector and the corresponding first output vector from the intersection to retrain the deep neural network in the source field Network model to obtain the network fault handling model of the target field.
参见图4所示,网络故障处理模型的构建方法还包括:S300求取目标领域的样本集与源领域的样本集的差集,基于差集优化目标领域的网络故障处理模型。As shown in FIG. 4, the method for constructing a network fault handling model further includes: S300 obtains the difference set between the sample set of the target domain and the sample set of the source domain, and optimizes the network fault handling model of the target domain based on the difference set.
在一个实施方式中,可以从差集中提取第二输入向量及对应的第二输出向量,重新训练目标领域的网络故障处理模型。In one embodiment, the second input vector and the corresponding second output vector can be extracted from the difference set, and the network fault handling model of the target domain can be retrained.
在另一个实施方式中,也可以从差集中提取第三输入向量,输入目标领域的网络故障处理模型得到第三输出向量;根据专家评估反馈结果对第三输入向量和第三输出向量进行修正后,重新训练目标领域的网络故障处理模型。In another embodiment, the third input vector can also be extracted from the difference set, and the third output vector can be obtained by inputting the network fault processing model of the target field; the third input vector and the third output vector are corrected according to the expert evaluation feedback result , Retrain the network fault handling model in the target domain.
其中,重新训练目标领域的网络故障处理模型包括:基于差集对目标领域的网络故障处理模型的神经元函数的权重系数进行修正,得到优化的目标领域的网络故障处理模型。Among them, retraining the network fault handling model of the target field includes: correcting the weight coefficient of the neuron function of the network fault handling model of the target field based on the difference set to obtain an optimized network fault handling model of the target field.
图6中交集矩阵I用于直接生成目标领域的深度神经网络模型中的拟合函数权重参数,图6中下部是源领域和目标领域的差集矩阵D,差集矩阵D是一个2行3列的矩阵,差集矩阵D用于优化目标领域的深度神经网络模型中的拟合函数权重参数。The intersection matrix I in Figure 6 is used to directly generate the weight parameters of the fitting function in the deep neural network model of the target field. The lower part of Figure 6 is the difference matrix D between the source field and the target field. The difference matrix D is a 2 row 3 The column matrix and the difference matrix D are used to optimize the weight parameters of the fitting function in the deep neural network model of the target field.
以图6为例,以说明通过差集数据x
22和y
22修正神经元函数f
22的权重系数w
22的过程。这个示例选取告警时间、告警类别和故障类别,并经过量化表示后构建输入向量,配置数据即配置方案量化表示 数据构建输出向量。本实施例中,配置方案量化表示数值1表示采用第1号配置方案,数置-1表示采用第2号配置方案。表1序号1这行数据对应输入向量x=(2,5,7),输出y=1,表示告警时间、告警类别和故障类别的量化表示数值分别为2、5和7时,配置方案量化表示值为1,这个输入向量和输出向量由深度学习神经网络模型神经元函数f
22通过公式y=f(x)=sgn(wx
T)进行拟合。
Take FIG. 6 as an example to illustrate the process of modifying the weight coefficient w 22 of the neuron function f 22 through the difference set data x 22 and y 22 . This example selects the alarm time, alarm category, and fault category, and constructs the input vector after quantified representation. The configuration data is the configuration plan quantified representation data to construct the output vector. In this embodiment, the quantification of the configuration plan indicates that the value 1 indicates that the first configuration plan is adopted, and the number set to -1 indicates that the second configuration plan is adopted. Table 1 Number 1 row of data corresponds to input vector x=(2,5,7), output y=1, indicating that the quantified values of alarm time, alarm category and fault category are 2, 5 and 7, respectively, and the configuration plan is quantified The expression value is 1. This input vector and output vector are fitted by the deep learning neural network model neuron function f 22 through the formula y=f(x)=sgn(wx T ).
作为一个示例,通过大量的类似于表2中序号1和序号2这样的样本数据,求得目标领域的深度神经网络模型的神经元函数的权重系数w。表2中交集数据表示交集中的样本数据,差集数据表示差集中的样本数据。表2中序号3对应的权重系数w=(1,0,1),满足sgn[(1,0,1)*(2,5,7)
T]=sgn(9)=1,sgn[(1,0,1)*(3,2,8)
T]=sgn(11)=1。表2中序号4和序号5对应差集数据,基于差集数据构建输入向量(5,7,3)和(8,3,7),输出y的值都是-1。将差集数据构建的输入向量和输出向量注入目标领域的深度神经网络模型,重新调整目标领域的深度神经网络模型的神经元函数的权重,得到表2中序号6对应的修正后的神经元函数的权重系数w=(1,-1,-1),这个权重系数满足sgn[(1,-1,-1)*(5,7,3)
T]=sgn(-5)=-1,sgn[[(1,-1,-1)*(8,3,7)
T]=sgn(-2)=-1。
As an example, the weight coefficient w of the neuron function of the deep neural network model of the target field is obtained through a large amount of sample data similar to the serial number 1 and the serial number 2 in Table 2. The intersection data in Table 2 represents the sample data in the intersection, and the difference data represents the sample data in the difference. The weight coefficient w = (1,0,1) corresponding to sequence number 3 in Table 2 satisfies sgn[(1,0,1)*(2,5,7) T ]=sgn(9)=1, sgn[( 1,0,1)*(3,2,8) T ]=sgn(11)=1. In Table 2, the sequence numbers 4 and 5 correspond to the difference set data. The input vectors (5,7,3) and (8,3,7) are constructed based on the difference set data, and the output y value is -1. Inject the input vector and output vector constructed by the difference data into the deep neural network model of the target field, readjust the weight of the neuron function of the deep neural network model of the target field, and obtain the corrected neuron function corresponding to the number 6 in Table 2. The weight coefficient w=(1,-1,-1), this weight coefficient satisfies sgn[(1,-1,-1)*(5,7,3) T ]=sgn(-5)=-1, sgn[[(1,-1,-1)*(8,3,7) T ]=sgn(-2)=-1.
表2 为基于差集修正神经元的权重系数的示例Table 2 is an example of correcting neuron weight coefficient based on difference set
基于差集数据不断修正优化目标领域的深度神经网络模型的神经元函数的权重参数,最后得到优化的目标领域的深度神经网络模型,实现光网络故障的自动愈和以及自动排除。修正优化后的神经元函数的权重参数保存在目标领域的深度神经网络模型的各神经元节点中,如图6右部分所示。Based on the difference set data, the weight parameters of the neuron function of the deep neural network model in the optimized target field are continuously revised, and finally the optimized deep neural network model in the target field is obtained, which realizes automatic recovery and automatic elimination of optical network faults. The weight parameters of the corrected and optimized neuron function are stored in each neuron node of the deep neural network model of the target field, as shown in the right part of Figure 6.
在上述描述中,步骤S300在前述实施例的步骤S200至S220的基础上,基于差集进一步优化目标领域的网络故障处理模型。In the foregoing description, step S300 is based on steps S200 to S220 of the foregoing embodiment, and further optimizes the network fault processing model of the target field based on the difference set.
与上述过程相似,步骤S300也可以在前述实施例的步骤S110至S130的基础上,基于差集进一步优化目标领域的网络故障处理模型,在此不再赘述。Similar to the foregoing process, step S300 can also be based on steps S110 to S130 of the foregoing embodiment to further optimize the network fault processing model of the target field based on the difference set, which will not be repeated here.
本发明实施例还提供一种网络故障处理方法,在前述的各实施例的基础上,网络故障处理方法包括:The embodiment of the present invention also provides a network fault processing method. Based on the foregoing embodiments, the network fault processing method includes:
S410获取目标网络的告警数据和故障数据,经过量化处理后输入网络故障处理模型,网络故障处理模型是使用前述的网络故障处理模型的构建方法得到的。S410 acquires alarm data and fault data of the target network, and inputs the network fault processing model after quantitative processing. The network fault processing model is obtained by using the aforementioned method for constructing the network fault processing model.
S420网络故障处理模型的输出向量下发到目标网络的相关设备。The output vector of the S420 network fault handling model is delivered to the relevant equipment of the target network.
本发明实施例基于光网络中源领域的深度神经网络模型,通过跨领域迁移学***台下发目标领域的设备,完成目标领域中设备恢复、导换、调参和重路由等操作,从而实现目标领域的网络故障自愈。The embodiment of the present invention is based on the deep neural network model of the source domain in the optical network, and obtains the network fault handling model of the target domain through cross-domain migration learning. When an alarm or fault occurs in the target domain, the network fault handling model automatically generates configuration data, and Through the management and control platform, the equipment in the target field is issued to complete operations such as equipment restoration, switching, parameter adjustment and rerouting in the target field, so as to realize the self-healing of network failures in the target field.
参见图7所示,本发明实施例还提供一种网络故障处理模型的构 建***,用于实现前述实施例网络故障处理模型的构建方法,网络故障处理模型的构建***包括获取模块100、处理模块200和构建模块300。As shown in FIG. 7, the embodiment of the present invention also provides a construction system of a network fault handling model, which is used to implement the construction method of the network fault handling model of the foregoing embodiment. The construction system of the network fault handling model includes an acquisition module 100 and a processing module. 200 and building block 300.
获取模块100用于基于网络中源领域的样本集102,获取或者建立源领域的深度神经网络模型。The acquiring module 100 is used to acquire or establish a deep neural network model of the source domain based on the sample set 102 of the source domain in the network.
在一种可能的实施方式中,获取模块100包括获取的源领域样本集102,以及基于源领域样本集102建立的源领域的深度神经网络模型。In a possible implementation, the acquisition module 100 includes the acquired source domain sample set 102 and a source domain deep neural network model established based on the source domain sample set 102.
在另一种可能的实施方式中,获取模块100包括源领域数据采集单元101、源领域样本集102和源领域的深度神经网络模型构建单元103。In another possible implementation, the acquisition module 100 includes a source domain data collection unit 101, a source domain sample set 102, and a source domain deep neural network model construction unit 103.
源领域数据采集单元101采集样本数据,并保存在源领域样本集102中,源领域的深度神经网络模型是源领域的深度神经网络模型构建单元103基于源领域样本集102构建的。The source domain data collection unit 101 collects sample data and saves it in the source domain sample set 102. The deep neural network model of the source domain is constructed by the source domain deep neural network model construction unit 103 based on the source domain sample set 102.
处理模块200用于建立网络中目标领域的样本集202,目标领域与源领域的样本集具有交集203,且均包括经过量化处理的告警数据、故障数据和配置数据。其中,处理模块200中的目标领域数据采集单元201采集样本数据,并保存在目标领域样本集202中。处理模块200还用于计算目标领域与源领域的样本集的重合率。The processing module 200 is used to establish a sample set 202 of the target domain in the network. The sample sets of the target domain and the source domain have an intersection 203, and both include quantitatively processed alarm data, fault data, and configuration data. Wherein, the target field data collection unit 201 in the processing module 200 collects sample data and saves it in the target field sample set 202. The processing module 200 is also used to calculate the coincidence rate of the sample sets of the target field and the source field.
构建模块300用于当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。The construction module 300 is used for constructing a network fault processing model of the target domain based on the deep neural network model of the source domain when the coincidence rate of the sample sets of the target domain and the source domain reaches a set threshold.
进一步的,构建模块300用于将源领域的深度神经网络模型作为目标领域的网络故障处理模型;还用于从交集中提取第一输入向量及对应的第一输出向量,重新训练源领域的深度神经网络模型,得到目 标领域的网络故障处理模型。Further, the construction module 300 is used to use the deep neural network model of the source domain as the network fault processing model of the target domain; it is also used to extract the first input vector and the corresponding first output vector from the intersection, and retrain the depth of the source domain. Neural network model to obtain the network fault handling model of the target field.
进一步的,处理模块200还用于求取目标领域的样本集与源领域的样本集的差集204。构建模块300用于基于差集204优化目标领域的网络故障处理模型。Further, the processing module 200 is also used to obtain the difference set 204 between the sample set in the target field and the sample set in the source field. The construction module 300 is used to optimize the network fault handling model of the target field based on the difference set 204.
进一步的,构建模块300还用于从差集204中提取第二输入向量及对应的第二输出向量,重新训练目标领域的网络故障处理模型。Further, the construction module 300 is also used for extracting the second input vector and the corresponding second output vector from the difference set 204, and retraining the network fault processing model of the target domain.
进一步的,构建模块300还用于从差集204中提取第三输入向量,输入源领域的深度神经网络模型得到第三输出向量;还用于根据专家评估反馈结果对第三输入向量和第三输出向量进行修正后,重新训练目标领域的网络故障处理模型。Further, the construction module 300 is also used to extract a third input vector from the difference set 204, and the deep neural network model of the input source domain is used to obtain the third output vector; it is also used to compare the third input vector and the third output vector according to the expert evaluation feedback result. After the output vector is corrected, the network fault handling model in the target field is retrained.
具体的,构建模块300用于基于差集204对目标领域的网络故障处理模型的神经元函数的权重系数进行修正,得到优化的目标领域的网络故障处理模型。Specifically, the construction module 300 is used to modify the weight coefficient of the neuron function of the network fault processing model of the target field based on the difference set 204 to obtain an optimized network fault processing model of the target field.
具体的,源领域的深度神经网络模型的输入向量包括经过量化处理的告警数据和故障数据,且输出向量为经过量化处理的配置数据。Specifically, the input vector of the deep neural network model of the source domain includes quantized alarm data and fault data, and the output vector is quantized configuration data.
参见图8所示,本发明实施例提供一种网络故障处理***,其包括输入控制模块400、模型处理模块500和输出控制模块600。Referring to FIG. 8, an embodiment of the present invention provides a network fault processing system, which includes an input control module 400, a model processing module 500, and an output control module 600.
输入控制模块400用于获取目标网络的告警数据和故障数据,并进行量化处理。The input control module 400 is used to obtain alarm data and fault data of the target network, and perform quantitative processing.
模型处理模块500用于存储前述的网络故障处理模型的构建***构建的网络故障处理模型,并将量化处理后的告警数据和故障数据输入网络故障处理模型,得到网络故障处理模型的输出向量。The model processing module 500 is used to store the network fault processing model constructed by the aforementioned network fault processing model construction system, and input the quantitatively processed alarm data and fault data into the network fault processing model to obtain the output vector of the network fault processing model.
输出控制模块600用于将网络故障处理模型的输出向量下发到目标网络的相关设备。The output control module 600 is used to deliver the output vector of the network fault handling model to related devices of the target network.
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者 其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时,全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line,DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够读取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,数字通用光盘(Digital Video Disc,DVD))或者半导体介质(例如,固态硬盘(Solid State Disk,SSD))等。In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website, computer, server or data center. The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center integrated with one or more available media. Available media can be magnetic media (for example, floppy disks, hard drives, tapes), optical media (for example, Digital Video Disc (DVD)) or semiconductor media (for example, Solid State Disk (SSD)), etc. .
本发明不局限于上述实施方式,对于本技术领域的普通技术人员来说,在不脱离本发明原理的前提下,还可以做出若干改进和润饰,这些改进和润饰也视为本发明的保护范围之内。本说明书中未作详细描述的内容属于本领域专业技术人员公知的现有技术。The present invention is not limited to the above-mentioned embodiments. For those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications are also regarded as the protection of the present invention. Within range. The content not described in detail in this specification belongs to the prior art known to those skilled in the art.
Claims (16)
- 一种网络故障处理模型的构建方法,其特征在于,其包括:A method for constructing a network fault handling model, which is characterized in that it includes:基于所述网络中源领域的样本集,获取或者建立源领域的深度神经网络模型;Acquiring or establishing a deep neural network model of the source domain based on the sample set of the source domain in the network;建立所述网络中目标领域的样本集,目标领域与源领域的样本集具有交集,且均包括经过量化处理的告警数据、故障数据和配置数据;Establishing a sample set of the target field in the network, the sample set of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data;当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。When the coincidence rate of the sample sets of the target domain and the source domain reaches the set threshold, the network fault handling model of the target domain is constructed based on the deep neural network model of the source domain.
- 如权利要求1所述的网络故障处理模型的构建方法,其特征在于:将所述源领域的深度神经网络模型作为所述目标领域的网络故障处理模型;或者,The method for constructing a network fault handling model according to claim 1, wherein the deep neural network model of the source domain is used as the network fault handling model of the target domain; or,从所述交集中提取第一输入向量及对应的第一输出向量,重新训练所述源领域的深度神经网络模型,得到所述目标领域的网络故障处理模型。Extracting the first input vector and the corresponding first output vector from the intersection, and retraining the deep neural network model of the source domain to obtain the network fault handling model of the target domain.
- 如权利要求1所述的网络故障处理模型的构建方法,其特征在于,所述方法还包括:求取所述目标领域的样本集与源领域的样本集的差集,基于所述差集优化所述目标领域的网络故障处理模型。The method for constructing a network fault handling model according to claim 1, wherein the method further comprises: obtaining the difference between the sample set of the target domain and the sample set of the source domain, and optimizing based on the difference set The network fault handling model of the target field.
- 如权利要求3所述的网络故障处理模型的构建方法,其特征在于:从所述差集中提取第二输入向量及对应的第二输出向量,重新训练所述目标领域的网络故障处理模型。3. The method for constructing a network fault processing model according to claim 3, wherein the second input vector and the corresponding second output vector are extracted from the difference set, and the network fault processing model of the target domain is retrained.
- 如权利要求3所述的网络故障处理模型的构建方法,其特征在于:从所述差集中提取第三输入向量,输入所述目标领域的网络故障处理模型并得到第三输出向量;3. The method for constructing a network fault processing model according to claim 3, characterized in that: extracting a third input vector from the difference set, inputting the network fault processing model of the target domain, and obtaining a third output vector;根据专家评估反馈结果对第三输入向量和第三输出向量进行修正后,重新训练所述目标领域的网络故障处理模型。After correcting the third input vector and the third output vector according to the expert evaluation feedback result, the network fault handling model in the target field is retrained.
- 如权利要求3所述的网络故障处理模型的构建方法,其特征在于:基于所述差集对所述目标领域的网络故障处理模型的神经元函数的权重系数进行修正,得到优化的所述目标领域的网络故障处理模型。The method for constructing a network fault processing model according to claim 3, wherein the weight coefficient of the neuron function of the network fault processing model of the target field is corrected based on the difference set to obtain the optimized target The network fault handling model of the domain.
- 如权利要求1所述的网络故障处理模型的构建方法,其特征在于:所述源领域的深度神经网络模型的输入向量包括所述经过量化处理的告警数据和故障数据,且输出向量为所述经过量化处理的配置数据。The method for constructing a network fault processing model according to claim 1, wherein the input vector of the deep neural network model of the source field includes the quantized alarm data and fault data, and the output vector is the Quantified configuration data.
- 一种网络故障处理方法,其特征在于,其包括:A network fault processing method, characterized in that it includes:获取目标网络的告警数据和故障数据,经过量化处理后输入网络故障处理模型,所述网络故障处理模型是使用如权利要求1至7任一项所述的网络故障处理模型的构建方法得到的;Obtain the alarm data and fault data of the target network, and input the network fault processing model after quantitative processing, the network fault processing model obtained by using the method for constructing a network fault processing model according to any one of claims 1 to 7;所述网络故障处理模型的输出向量下发到目标网络的相关设备。The output vector of the network fault handling model is delivered to the relevant equipment of the target network.
- 一种网络故障处理模型的构建***,其特征在于,其包括:A construction system for a network fault handling model, which is characterized in that it includes:获取模块,其用于基于所述网络中源领域的样本集,获取或者建立源领域的深度神经网络模型;An acquisition module, which is used to acquire or establish a deep neural network model of the source domain based on the sample set of the source domain in the network;处理模块,其用于建立所述网络中目标领域的样本集,目标领域与源领域的样本集具有交集,且均包括经过量化处理的告警数据、故障数据和配置数据;以及计算目标领域与源领域的样本集的重合率;A processing module, which is used to establish a sample set of the target field in the network, the sample set of the target field and the source field have an intersection, and both include quantified alarm data, fault data, and configuration data; and calculate the target field and the source The coincidence rate of the sample set of the field;构建模块,其用于当目标领域与源领域的样本集的重合率达到设定的阈值时,基于源领域的深度神经网络模型,构建目标领域的网络故障处理模型。The construction module is used to construct a network fault handling model of the target domain based on the deep neural network model of the source domain when the coincidence rate of the sample sets of the target domain and the source domain reaches a set threshold.
- 如权利要求9所述的网络故障处理模型的构建***,其特征在于:所述构建模块用于将所述源领域的深度神经网络模型作为所述目标领域的网络故障处理模型;还用于从所述交集中提取第一输入 向量及对应的第一输出向量,重新训练所述源领域的深度神经网络模型,得到所述目标领域的网络故障处理模型。The network fault processing model construction system according to claim 9, wherein the construction module is used to use the deep neural network model of the source domain as the network fault handling model of the target domain; The first input vector and the corresponding first output vector are extracted from the intersection, and the deep neural network model of the source domain is retrained to obtain the network fault handling model of the target domain.
- 如权利要求9所述的网络故障处理模型的构建***,其特征在于:所述处理模块还用于求取所述目标领域的样本集与源领域的样本集的差集;10. The network fault processing model construction system according to claim 9, wherein the processing module is further used to obtain the difference between the sample set of the target domain and the sample set of the source domain;所述构建模块还用于基于所述差集优化所述目标领域的网络故障处理模型。The construction module is also used for optimizing the network fault processing model of the target domain based on the difference set.
- 如权利要求11所述的网络故障处理模型的构建***,其特征在于:所述构建模块用于从所述差集中提取第二输入向量及对应的第二输出向量,重新训练所述目标领域的网络故障处理模型。The network fault handling model construction system of claim 11, wherein the construction module is used to extract a second input vector and a corresponding second output vector from the difference set, and retrain the target domain Network fault handling model.
- 如权利要求11所述的网络故障处理模型的构建***,其特征在于:所述构建模块用于从所述差集中提取第三输入向量,输入所述目标领域的网络故障处理模型得到第三输出向量;还用于根据专家评估反馈结果对第三输入向量和第三输出向量进行修正后,重新训练所述目标领域的网络故障处理模型。The network fault processing model construction system of claim 11, wherein the building module is used to extract a third input vector from the difference set, and input the network fault processing model of the target domain to obtain a third output Vector; also used to modify the third input vector and the third output vector according to the expert evaluation feedback result, and then retrain the network fault handling model in the target field.
- 如权利要求11所述的网络故障处理模型的构建***,其特征在于:所述构建模块用于基于所述差集对所述目标领域的网络故障处理模型的神经元函数的权重系数进行修正,得到优化的所述目标领域的网络故障处理模型。The network fault processing model construction system according to claim 11, wherein the building module is used to modify the weight coefficients of the neuron function of the network fault processing model of the target field based on the difference set, An optimized network fault handling model of the target field is obtained.
- 如权利要求9所述的网络故障处理模型的构建***,其特征在于:所述源领域的深度神经网络模型的输入向量包括所述经过量化处理的告警数据和故障数据,且输出向量为所述经过量化处理的配置数据。The network fault processing model construction system of claim 9, wherein the input vector of the deep neural network model of the source field includes the quantized alarm data and fault data, and the output vector is the Quantified configuration data.
- 一种网络故障处理***,其特征在于,其包括:A network fault processing system, characterized in that it includes:输入控制模块,其用于获取目标网络的告警数据和故障数据,并 进行量化处理;Input control module, which is used to obtain alarm data and fault data of the target network and perform quantitative processing;模型处理模块,其用于存储由权利要求9至15任一所述的网络故障处理模型的构建***构建的网络故障处理模型,并将量化处理后的告警数据和故障数据输入所述网络故障处理模型,得到所述网络故障处理模型的输出向量;A model processing module, which is used to store the network fault processing model constructed by the network fault processing model construction system of any one of claims 9 to 15, and input the quantitatively processed alarm data and fault data into the network fault processing Model to obtain the output vector of the network fault handling model;输出控制模块,其用于将所述网络故障处理模型的输出向量下发到目标网络的相关设备。The output control module is used to deliver the output vector of the network fault handling model to the relevant equipment of the target network.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910128532.6A CN111600734B (en) | 2019-02-21 | 2019-02-21 | Network fault processing model construction method, fault processing method and system |
CN201910128532.6 | 2019-02-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020168676A1 true WO2020168676A1 (en) | 2020-08-27 |
Family
ID=72143395
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/096623 WO2020168676A1 (en) | 2019-02-21 | 2019-07-19 | Method for constructing network fault handling model, fault handling method and system |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN111600734B (en) |
WO (1) | WO2020168676A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112330060A (en) * | 2020-11-25 | 2021-02-05 | 新智数字科技有限公司 | Equipment fault prediction method and device, readable storage medium and electronic equipment |
CN112752172A (en) * | 2020-12-15 | 2021-05-04 | 烽火通信科技股份有限公司 | Optical channel fault diagnosis method and system based on transfer learning |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107491792A (en) * | 2017-08-29 | 2017-12-19 | 东北大学 | Feature based maps the electric network fault sorting technique of transfer learning |
CN108548671A (en) * | 2018-03-12 | 2018-09-18 | 南京航空航天大学 | A kind of method for diagnosing faults of the shafting rotating speed great fluctuation process based on autocoder |
CN108549907A (en) * | 2018-04-11 | 2018-09-18 | 武汉大学 | A kind of data verification method based on multi-source transfer learning |
US20180268296A1 (en) * | 2016-06-02 | 2018-09-20 | Tencent Technology (Shenzhen) Company Limited | Machine learning-based network model building method and apparatus |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101715149B (en) * | 2009-07-21 | 2013-01-23 | 北京邮电大学 | Method and device for restoring parallel cross-domain fault of multilayer and multi-domain distributed optical network |
CN101794396B (en) * | 2010-03-25 | 2012-12-26 | 西安电子科技大学 | System and method for recognizing remote sensing image target based on migration network learning |
US9009215B2 (en) * | 2013-03-15 | 2015-04-14 | Wandisco, Inc. | Methods, devices and systems for dynamically managing memberships in replicated state machines within a distributed computing environment |
CN105300693B (en) * | 2015-09-25 | 2016-10-12 | 东南大学 | A kind of Method for Bearing Fault Diagnosis based on transfer learning |
WO2017145358A1 (en) * | 2016-02-26 | 2017-08-31 | 三菱電機株式会社 | Wireless communication apparatus and number-of-transmission-streams determination method |
US10511613B2 (en) * | 2017-01-24 | 2019-12-17 | Nec Corporation | Knowledge transfer system for accelerating invariant network learning |
CN107341146B (en) * | 2017-06-23 | 2020-08-04 | 上海交大知识产权管理有限公司 | Migratable spoken language semantic analysis system based on semantic groove internal structure and implementation method thereof |
CN107679580B (en) * | 2017-10-21 | 2020-12-01 | 桂林电子科技大学 | Heterogeneous migration image emotion polarity analysis method based on multi-mode depth potential correlation |
CN107958286A (en) * | 2017-11-23 | 2018-04-24 | 清华大学 | A kind of depth migration learning method of field Adaptive Networking |
CN108304876B (en) * | 2018-01-31 | 2021-07-06 | 国信优易数据股份有限公司 | Classification model training method and device and classification method and device |
-
2019
- 2019-02-21 CN CN201910128532.6A patent/CN111600734B/en active Active
- 2019-07-19 WO PCT/CN2019/096623 patent/WO2020168676A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180268296A1 (en) * | 2016-06-02 | 2018-09-20 | Tencent Technology (Shenzhen) Company Limited | Machine learning-based network model building method and apparatus |
CN107491792A (en) * | 2017-08-29 | 2017-12-19 | 东北大学 | Feature based maps the electric network fault sorting technique of transfer learning |
CN108548671A (en) * | 2018-03-12 | 2018-09-18 | 南京航空航天大学 | A kind of method for diagnosing faults of the shafting rotating speed great fluctuation process based on autocoder |
CN108549907A (en) * | 2018-04-11 | 2018-09-18 | 武汉大学 | A kind of data verification method based on multi-source transfer learning |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112330060A (en) * | 2020-11-25 | 2021-02-05 | 新智数字科技有限公司 | Equipment fault prediction method and device, readable storage medium and electronic equipment |
CN112330060B (en) * | 2020-11-25 | 2024-01-12 | 新奥新智科技有限公司 | Equipment fault prediction method and device, readable storage medium and electronic equipment |
CN112752172A (en) * | 2020-12-15 | 2021-05-04 | 烽火通信科技股份有限公司 | Optical channel fault diagnosis method and system based on transfer learning |
CN112752172B (en) * | 2020-12-15 | 2022-03-25 | 烽火通信科技股份有限公司 | Optical channel fault diagnosis method and system based on transfer learning |
Also Published As
Publication number | Publication date |
---|---|
CN111600734B (en) | 2021-11-02 |
CN111600734A (en) | 2020-08-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN113053115B (en) | Traffic prediction method based on multi-scale graph convolution network model | |
WO2019091020A1 (en) | Weight data storage method, and neural network processor based on method | |
CN111030861B (en) | Edge calculation distributed model training method, terminal and network side equipment | |
CN112784964A (en) | Image classification method based on bridging knowledge distillation convolution neural network | |
WO2020168676A1 (en) | Method for constructing network fault handling model, fault handling method and system | |
CN106909728B (en) | FPGA interconnection resource configuration generation method based on reinforcement learning | |
CN113132232B (en) | Energy route optimization method | |
WO2020168675A1 (en) | Sample data processing method, and system and apparatus | |
CN116644804B (en) | Distributed training system, neural network model training method, device and medium | |
CN113780470A (en) | Graph classification method based on self-adaptive multi-channel cross graph convolution network | |
CN112215353A (en) | Channel pruning method based on variational structure optimization network | |
CN111597276B (en) | Entity alignment method, device and equipment | |
CN112733863A (en) | Image feature extraction method, device, equipment and storage medium | |
Bi et al. | Knowledge transfer for out-of-knowledge-base entities: Improving graph-neural-network-based embedding using convolutional layers | |
CN115115113A (en) | Equipment fault prediction method and system based on graph attention network relation embedding | |
CN112529148B (en) | Intelligent QoS inference method based on graph neural network | |
CN114205251A (en) | Switch link resource prediction method based on space-time characteristics | |
Li | Computer network connection enhancement optimization algorithm based on convolutional neural network | |
Liu et al. | A new perspective to algebraic characterization on controllability of multiagent systems | |
WO2019200548A1 (en) | Network model compiler and related product | |
CN115294402B (en) | Semi-supervised vehicle classification method based on redundancy elimination multi-stage hybrid training | |
CN114637720B (en) | On-chip optical network mapping method based on growth type annular SOM neural network | |
WO2023015674A1 (en) | Multi-bit-width quantization method for deep convolutional neural network | |
RU2780710C1 (en) | Method, system, and apparatus for processing sample data | |
Shang et al. | DeepAutonet: self-driving reconfigurable HPC system with deep reinforcement learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19915911 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19915911 Country of ref document: EP Kind code of ref document: A1 |