CN114818895A - Model training and abnormal data identification method, device, equipment and medium - Google Patents
Model training and abnormal data identification method, device, equipment and medium Download PDFInfo
- Publication number
- CN114818895A CN114818895A CN202210399453.0A CN202210399453A CN114818895A CN 114818895 A CN114818895 A CN 114818895A CN 202210399453 A CN202210399453 A CN 202210399453A CN 114818895 A CN114818895 A CN 114818895A
- Authority
- CN
- China
- Prior art keywords
- samples
- existing
- data
- characteristic values
- groups
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Housings And Mounting Of Transformers (AREA)
Abstract
The invention discloses a model training and abnormal data identification method, device, equipment and medium. The method comprises the following steps: determining an expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data; the existing characteristic values of different groups of samples are the content values of at least two gases in transformer oil collected at different time points; and training an abnormal data identification model according to the at least two groups of existing characteristic values of the samples, the expanded characteristic values corresponding to the at least two groups of existing characteristic values of the samples and the abnormal data labels associated with the oil chromatogram data of the samples, wherein the abnormal data identification model is used for identifying abnormal data in the oil chromatogram data. According to the technical scheme provided by the embodiment of the invention, the trained abnormal data identification model has higher accuracy, and further, the abnormal data in the oil chromatogram data can be accurately identified based on the model.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a method, a device, equipment and a medium for model training and abnormal data identification.
Background
With the development of transformer technology, oil-immersed transformers have been widely used, and the bodies of the oil-immersed transformers are often installed in oil tanks filled with transformer oil, and the oil tanks are welded by steel plates. In an application scene of the oil-immersed transformer, the oil color collecting device needs to be used for collecting oil color spectrum data of the oil-immersed transformer, but when the oil color collecting device breaks down, the collected oil color spectrum data of the oil-immersed transformer is abnormal.
Therefore, how to perform anomaly detection on oil chromatographic data of the oil-immersed transformer and provide higher-quality oil chromatographic data is a problem to be solved urgently at present.
Disclosure of Invention
The embodiment of the invention provides a method, a device, equipment and a medium for model training and abnormal data identification, wherein the trained abnormal data identification model has higher accuracy, and further more accurate identification of abnormal data in oil chromatogram data based on the model can be realized.
In a first aspect, an embodiment of the present invention provides a model training method, including:
determining an expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data; the existing characteristic values of different groups of samples are the content values of at least two gases in transformer oil collected at different time points;
and training an abnormal data identification model according to the at least two groups of existing characteristic values of the samples, the expanded characteristic values corresponding to the at least two groups of existing characteristic values of the samples and the abnormal data labels associated with the oil chromatogram data of the samples, wherein the abnormal data identification model is used for identifying abnormal data in the oil chromatogram data.
In a second aspect, an embodiment of the present invention further provides an abnormal data identification method based on oil chromatography data, including:
acquiring at least two groups of existing target characteristic values in target oil chromatographic data, wherein the existing characteristic values of different groups of targets are content values of at least two gases in transformer oil collected at different time points;
inputting at least two groups of existing target characteristic values in the target oil chromatogram data into a trained abnormal data identification model to obtain abnormal data in the at least two groups of existing target characteristic values;
the abnormal data identification model is obtained by training according to the model training method provided by any embodiment of the invention.
In a third aspect, an embodiment of the present invention further provides a model training apparatus, including:
the determining module is used for determining the expansion characteristic values corresponding to the existing characteristic values of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data; the existing characteristic values of different groups of samples are the content values of at least two gases in transformer oil collected at different time points;
and the training module is used for training an abnormal data identification model according to the existing characteristic values of the at least two groups of samples, the expanded characteristic values corresponding to the existing characteristic values of the at least two groups of samples and the abnormal data labels associated with the sample oil chromatographic data, and is used for identifying abnormal data in the oil chromatographic data.
In a fourth aspect, an embodiment of the present invention further provides an abnormal data identification device based on oil chromatography data, including:
the acquisition module is used for acquiring at least two groups of target existing characteristic values in target oil chromatographic data, wherein the different groups of target existing characteristic values are content values of at least two gases in transformer oil acquired at different time points;
the obtaining module is used for inputting at least two groups of target existing characteristic values in the target oil chromatographic data into a trained abnormal data identification model to obtain abnormal data in the at least two groups of target existing characteristic values; the abnormal data identification model is obtained by training according to the model training method provided by any embodiment of the invention.
In a fifth aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a model training method as provided in any embodiment of the invention, and/or an abnormal data identification method based on oil chromatography data.
In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. Wherein the program when executed by the processor implements a model training method as provided by any of the embodiments of the present invention, and/or an abnormal data identification method based on oil chromatography data.
According to the scheme provided by the embodiment of the invention, the extended characteristic values are determined according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data, and then the abnormal data identification model is trained according to the existing characteristic values of at least two groups of samples, the corresponding extended characteristic values and the abnormal data labels associated with the sample oil chromatographic data, so that the model for identifying the abnormal data in the oil chromatographic data can be obtained. Furthermore, at least two groups of existing characteristic values of the target in the target oil chromatogram data are obtained and input into the trained abnormal data identification model, so that abnormal data in the existing characteristic values can be obtained, and the abnormal data can be identified. Through the mode, the trained abnormal data identification model is higher in accuracy, so that the abnormal data in the oil chromatogram data can be accurately identified based on the model, and the oil chromatogram data with higher quality can be obtained.
Drawings
Fig. 1A is a flowchart of a model training method according to an embodiment of the present invention;
FIG. 1B is a schematic diagram of a model structure according to an embodiment of the present invention;
FIG. 2 is a flowchart of a model training method according to a second embodiment of the present invention;
FIG. 3 is a flowchart of a model training method according to a third embodiment of the present invention;
fig. 4 is a flowchart of an abnormal data identification method based on oil chromatographic data according to a fourth embodiment of the present invention;
fig. 5 is a block diagram of a model training apparatus according to a fifth embodiment of the present invention;
fig. 6 is a block diagram of an abnormal data identification apparatus based on oil chromatogram data according to a sixth embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to a seventh embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings, not all of them.
Example one
Fig. 1A is a flowchart of a model training method according to a first embodiment of the present invention, and fig. 1B is a schematic diagram of a model structure according to a first embodiment of the present invention. The embodiment can be applied to the condition of training the abnormal data identification model, wherein the abnormal data in the oil chromatogram data is caused by the fault of the oil color acquisition device. The method may be performed by a model training apparatus, which may be implemented in software and/or hardware. As shown in fig. 1A, the method specifically includes:
s101, determining an expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data.
The sample oil chromatographic data refers to data comprising at least two groups of existing sample characteristic values and extended characteristic values corresponding to the existing characteristic values, and the label information of abnormal data in the sample oil chromatographic data is known. The existing characteristic value of the sample refers to content data of at least two gases dissolved in transformer oil acquired by the oil color acquisition device. The expansion characteristic value refers to a data characteristic value determined by expansion according to an existing characteristic value. The existing characteristic values of one group of samples refer to the content values of at least two gases in transformer oil collected at the same time, and the existing characteristic values of different groups of samples refer to the content values of at least two gases in transformer oil collected at different time points, wherein the unit of the gas content is expressed as concentration (ppm). The gas contained in the transformer oil may be at least two of hydrogen (h2), methane (ch4), ethylene (c2h4), acetylene (c2h2), carbon monoxide (co), carbon dioxide (co2), oxygen (o2), hydrocarbons (thc, total hydrocarbons), and ethane (c2h 6).
Optionally, the existing characteristic values in the sample oil chromatographic data may be periodically collected, that is, at least two gases in the transformer oil are collected at intervals of a preset time to determine the content values thereof, or the collection of the gases in the transformer oil is triggered once to determine the content values thereof when the relevant personnel specifies or meets preset conditions.
Optionally, after determining the existing characteristic values of at least two groups of samples in the sample oil chromatographic data, calculating the existing characteristic values based on a preset rule to determine an extended characteristic value; or inputting the existing characteristic values into a preset extended value determination model, and outputting the extended characteristic values corresponding to the existing characteristic values of each group, namely determining the extended characteristic values corresponding to the existing characteristic values of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data.
For example, referring to table 1, if three groups of existing sample characteristic values are stored in the sample oil chromatogram data, and the existing characteristic values of different groups of samples include hydrogen (h2), methane (ch4), ethylene (c2h4), acetylene (c2h2), carbon monoxide (co), carbon dioxide (co2), oxygen (o2), hydrocarbons (thc, total hydrocarbons), and ethane (c2h6), the existing characteristic values of the samples may be shown in the form of the following table:
TABLE 1 sample existing eigenvalues
S102, training an abnormal data identification model according to the existing characteristic values of the at least two groups of samples, the expanded characteristic values corresponding to the existing characteristic values of the at least two groups of samples and the abnormal data labels related to the oil chromatogram data of the samples, and identifying abnormal data in the oil chromatogram data.
The abnormal data refers to data in which the value of the characteristic data changes abruptly. The abnormal data label is a label which is calibrated for each data in the oil chromatogram data in advance and represents whether each group of characteristic value data is abnormal data or normal data.
Alternatively, the existing characteristic value and the corresponding extended characteristic value collected at each time may be used as a set of characteristic value data, for example, after the existing characteristic value of each set of samples, the corresponding extended characteristic value is added (for example, a new row of values of the supplementary characteristic value may be added in table 1), so as to form new sample oil chromatographic data. Therefore, at least two groups of characteristic value data containing the existing characteristic values and the expanded characteristic values can be obtained, further, the characteristic value data of the preset group number (at least two groups) is taken as a whole each time, the abnormal data identification model is input, the abnormal characteristic value data in the preset group number and the corresponding abnormal probability are obtained, specifically, whether the last group of data is abnormal or not can be determined according to the previous groups of data, and the characteristic value data of all the preset groups can be analyzed to determine the abnormal characteristic value data, for example, when the preset groups are 5 groups, whether the characteristic value data of the 5 th group is abnormal or not can be determined according to the characteristic value data of the first 4 groups, the abnormal data can be directly used for identifying the model, and analyzing the trend of the 5 groups of characteristic value data values to determine abnormal characteristic value data, wherein the other groups of data except the abnormal data group are normal data.
Optionally, after determining the abnormal characteristic value data according to the abnormal data identification model, for each group of characteristic value data, a prediction result of whether the group of data is abnormal or not may be obtained, according to the prediction result, the abnormal data label associated with the sample oil chromatogram data is compared, loss of the prediction result and the actual result is determined, finally, based on the loss, a parameter of the training abnormal data identification model is adjusted, that is, the training abnormal data identification model is used, and the trained abnormal data identification model may be used to identify abnormal data in the oil chromatogram data.
Optionally, the abnormal data identification model is constructed based on a Long Short-Term Memory LSTM (Long Short-Term Memory) network. For example, referring to fig. 1B, the input layer of the LSTM network may be set to 8 time-sequentially connected networks (containing 8 σ), the output layer may be set to 2 nodes, the hidden layer may be set to 32 nodes, and the drop parameter (Dropout) may be set to 0.25.
The method has the advantages that the abnormal data recognition model is constructed by utilizing the classical LSTM neural network model, the abnormal data recognition model with better recognition effect is trained, and therefore the abnormal data recognition accuracy is higher.
According to the scheme provided by the embodiment of the invention, the extended characteristic values are determined according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data, and then the abnormal data identification model is trained according to the existing characteristic values of at least two groups of samples, the corresponding extended characteristic values and the abnormal data labels associated with the sample oil chromatographic data, so that the model for identifying the abnormal data in the oil chromatographic data can be obtained.
Optionally, training an abnormal data recognition model according to the existing characteristic values of the at least two groups of samples, the extended characteristic values corresponding to the existing characteristic values of the at least two groups of samples, and the abnormal data labels associated with the sample oil chromatography data, includes: taking the existing characteristic value of each group of samples and the expanded characteristic value corresponding to the existing characteristic value of the group of samples as a group of data sets; combining different groups of data sets according to the acquisition time of the existing characteristic values of each group of samples to obtain a training set, a verification set and a test set; and training an abnormal data identification model based on the training set, the verification set and the test set and the abnormal data label associated with the sample oil chromatographic data.
Specifically, after the existing eigenvalue and the corresponding extended eigenvalue acquired at each moment are used as a group of eigenvalue data, the eigenvalue data of a preset group number (e.g., 5) spaced by a preset time period or adjacent acquisition time according to a preset rule may be used as a group of data sets, the data sets are further sorted based on the acquisition time of the last group of data in each group of data sets, and then the sorted data sets are combined according to a preset proportional relationship, such as a proportional relationship of 7:2:1 between the training set, the verification set and the test set, to obtain the training set, the verification set and the test set. And finally, adjusting relevant parameters of the abnormal data identification model by using the training set and the abnormal data labels, verifying the identification effect of the abnormal data identification model by using the verification set and the abnormal data labels after the training is finished, and testing the identification accuracy of the abnormal data identification model by using the test set and the abnormal data labels after the verification is passed.
The benefits of this arrangement are: the abnormal data identification model is trained by combining at least two groups of existing characteristic values and extended characteristic values to determine at least two groups of data sets and further dividing the data sets into a training set, a verification set and a test set, so that the abnormal data identification model obtained by training has higher identification accuracy on abnormal data in the oil chromatogram data.
Example two
Fig. 2 is a flowchart of a model training method according to a second embodiment of the present invention, and this embodiment further explains in detail "determining an extended eigenvalue corresponding to an existing eigenvalue of each group of samples according to at least two groups of existing eigenvalues of samples in sample oil chromatogram data" based on the above embodiment, as shown in fig. 2, the method specifically includes:
s201, determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the correlation among the existing characteristic values of each group of samples in the sample oil chromatographic data.
Wherein, the correlation between the existing characteristic values refers to the numerical relationship between at least two existing characteristic values. The first extended characteristic value is an extended characteristic value corresponding to the existing characteristic value of the gas acquired at the same time.
Alternatively, the first extended characteristic value of the sample oil chromatogram data may be determined by correlation between the content values of the different gases collected at each time point. Specifically, the numerical relationship between the content values of different gases collected at each time point can be calculated according to a preset calculation rule, so as to determine a first extended characteristic value; the first extended characteristic value can also be determined by performing a correlation analysis of the content values of the different gases collected at each time point using classical test methods.
Optionally, according to a preset calculation rule, a numerical relationship between content values of different gases collected at each time point is calculated, so that there are many ways to determine the first extended characteristic value, specifically, the first extended characteristic value of the sample oil chromatogram data may be determined according to a ratio of one of the gases collected at each time point to one or more other gas content values, or a ratio of one or more other gases collected at each time point to one of the gases, or the first extended characteristic value of the sample oil chromatogram data may be determined according to a difference between any two gas content values collected at each time point.
For example, the ratio of the content values of acetylene (c2h2) and ethylene (c2h4) collected at each time point may be used as a first extended value corresponding to the set of existing eigenvalues at the time point, that is, the first extended value of each set of existing eigenvalues is determined according to the formula c2h2/c2h 4; or the ratio of the content values of methane (ch4) and hydrocarbon (thc) collected at each time point can be used as a first extended value corresponding to the group of existing eigenvalues at the time point, namely, the first extended value of each group of existing eigenvalues is determined according to the formula ch 4/thc; the first extended value of each set of existing eigenvalues can also be determined according to the formula (ch4+ c2h4)/thc, ch4/(h2+ thc), ch4/(ch4+ c2h4+ c2h2) or ch4-c2h4, wherein ch4, c2h4, thc, h2 and c2h2 represent the content values of methane, ethylene, hydrocarbons, hydrogen and acetylene, respectively.
Alternatively, the first augmented eigenvalue may be determined by performing a correlation analysis of the content values of the different gases collected at each time point using classical non-parametric rank-sum tests. Correspondingly, according to the correlation between the existing characteristic values of each group of samples in the sample oil chromatogram data, determining a first extended characteristic value corresponding to the existing characteristic value of each group of samples, which comprises the following steps: analyzing the correlation between the existing characteristic values of each group of samples in the sample oil chromatographic data by adopting a rank sum check method to obtain at least two gas correlation values corresponding to the existing characteristic values of each group of samples; and determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to at least two gas correlation values corresponding to the existing characteristic values of each group of samples.
Optionally, for an existing characteristic value acquired at the same time point in the sample oil chromatography data, a rank sum check method may be adopted, for each gas, the existing characteristic value of the gas is analyzed, and the correlation between the existing characteristic value of the gas and the existing characteristic values of other gases is obtained, specifically, the deviation between each gas and the existing characteristic values of other gases may be determined first, then all the deviations are summed to obtain a total score of the gas, the score of the total score is used as the correlation value of the gas, and so on, so as to obtain the correlation values of at least two gases.
Optionally, after obtaining the correlation values of the at least two gases, at least one correlation value may be further selected from the obtained correlation values of the gases as the first extended characteristic value of the sample oil chromatogram data, specifically, the correlation values of the at least two gases may be sorted according to the size of the score values, and one or more correlation values with a higher score may be selected as the first extended characteristic value of the sample oil chromatogram data.
The benefits of this arrangement are: by adopting a classical calibration algorithm, at least two gas correlation values corresponding to the existing characteristic values of each group of samples are determined firstly, and then a first expansion characteristic value is determined, so that the determined sample oil chromatographic data can better represent the state of the transformer oil, and an abnormal data identification model trained by the data is more accurate and reasonable.
S202, determining a second expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the corresponding content value of the same gas in the existing characteristic values of different groups of samples.
The second extended characteristic value refers to an extended characteristic value corresponding to an existing characteristic value of the gas acquired at different moments.
Optionally, after determining the corresponding content value of the same gas in the existing characteristic values of each group of samples, calculating the corresponding content value of the same gas in the existing characteristic values of each group of samples according to a preset calculation rule, and determining a corresponding second extended characteristic value; for each gas, the corresponding content value of the gas in the existing characteristic values of each group of samples can be input into a pre-trained complementary feature determination model, and the corresponding complementary feature is output, namely, the second complementary feature is determined.
S203, training an abnormal data identification model according to the existing characteristic values of the at least two groups of samples, the expanded characteristic values corresponding to the existing characteristic values of the at least two groups of samples and the abnormal data labels associated with the oil chromatogram data of the samples, and identifying abnormal data in the oil chromatogram data.
According to the scheme provided by the embodiment of the invention, the first extended characteristic value is determined according to the correlation among the existing characteristic values of each group of samples in the sample oil chromatogram data, the second extended characteristic value is determined according to the corresponding content values of the same gas in the existing characteristic values of different groups of samples, the abnormal data identification model is trained by further combining the abnormal data label associated with the sample oil chromatogram data, and an implementable mode for determining the supplement characteristic value is provided.
EXAMPLE III
Fig. 3 is a flowchart of a model training method according to a third embodiment of the present invention, and this embodiment further explains in detail "determining a second extended eigenvalue corresponding to an existing eigenvalue of each group of samples according to a corresponding content value of the same gas in existing eigenvalues of different groups of samples" based on the above embodiment, as shown in fig. 3, the method specifically includes:
s301, determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the correlation among the existing characteristic values of each group of samples in the sample oil chromatogram data.
S302, determining the absolute growth amount and/or the relative growth rate of the target gas according to the corresponding content values of the target gas in the existing characteristic values of the first group of samples and the existing characteristic values of the second group of samples, and taking the absolute growth amount and/or the relative growth rate as a second expansion characteristic value corresponding to the existing characteristic values of the second group of samples.
The target gas is a gas whose content value needs to be determined to determine the supplementary characteristic value. The existing characteristic values of the first group of samples are acquired at a time point earlier than the existing characteristic values of the second group of samples. The absolute increase amount is obtained by subtracting the existing characteristic values of the same gas collected at different moments. The relative growth rate is the ratio of the absolute growth amount to the time difference of the acquisition of the existing characteristic values of at least two groups of samples.
Optionally, all the gases contained in the transformer oil may be sequentially used as target gases, or a part of the gases contained in the transformer oil may be selected as target gases according to a preset rule, that is, the target gases are determined.
Optionally, after the target gas is determined, for each target gas, the content values of the target gas in the existing characteristic values of the first group and the second group of samples may be determined, and the difference between the determined two content values is made to determine the absolute increase amount of the target gas, which is used as a second extended characteristic value corresponding to the existing characteristic values of the second group of samples; or after determining the absolute increase of the target gas, further determining the acquisition time difference of the existing characteristic values of the first group and the second group of samples, and taking the ratio of the absolute increase to the acquisition time difference, namely the relative increase rate, as a second extended characteristic value corresponding to the existing characteristic value of the second group of samples; the absolute growth amount and the relative growth rate may also be used as a second extended eigenvalue corresponding to the existing eigenvalue of the second group of samples, which is not limited in this embodiment.
Optionally, after the target gas is determined, the relative increase amount of the target gas may be further determined as a second extended eigenvalue corresponding to the existing eigenvalue of the second group of samples, specifically, if the acquisition times of the existing eigenvalues of the first group and the second group of samples are respectively p time and q time, the difference between the existing eigenvalues of the same gas acquired at p time and q time may be calculated first, and then the ratio between the difference and the existing eigenvalue of the gas at q time is calculated as the relative increase amount, and further, the relative increase amount is also used as a second extended eigenvalue corresponding to the existing eigenvalue of the second group of samples (i.e., the existing eigenvalue of the gas acquired at q time).
S303, training an abnormal data identification model according to the existing characteristic values of the at least two groups of samples, the expanded characteristic values corresponding to the existing characteristic values of the at least two groups of samples and the abnormal data labels associated with the oil chromatogram data of the samples, and identifying the abnormal data in the oil chromatogram data.
According to the scheme provided by the embodiment of the invention, after the first extended characteristic value is determined, the absolute increase amount and/or the relative increase rate of the target gas are determined according to the corresponding content values of the target gas in the existing characteristic values of the first group and the second group of samples, the absolute increase amount and/or the relative increase rate are/is taken as the second extended characteristic value, and finally the abnormal data identification model is trained by combining the abnormal data label associated with the sample oil chromatogram data.
Example four
Fig. 4 is a flowchart of an abnormal data identification method based on oil chromatogram data according to a fourth embodiment of the present invention, where this embodiment is applicable to the case of identifying abnormal data caused by a fault in an oil color collection device, and is particularly applicable to the case of identifying abnormal data by using an abnormal data identification model trained in the foregoing embodiment. As shown in fig. 4, the method for identifying abnormal data based on oil chromatogram data provided in this embodiment specifically includes:
s401, obtaining at least two groups of existing characteristic values of the target in the target oil chromatographic data.
Wherein the existing characteristic values of different groups of targets are the content values of at least two gases in the transformer oil collected at different time points. The target oil chromatographic data refers to oil chromatographic data which needs to detect whether abnormal data exists or not. The label information of the abnormal data in the target oil chromatogram data is unknown. The target existing characteristic value refers to an existing characteristic value which is determined from the existing characteristic values and needs to be input into the model for prediction.
Optionally, the target oil chromatogram data may be determined by manual selection, or the oil chromatogram data satisfying the condition may be determined as the target oil chromatogram data when it is detected that the oil chromatogram data satisfies a certain preset condition.
Optionally, after the target oil chromatogram data is determined, the existing characteristic values of all groups included in the target oil chromatogram data may be used as the target existing characteristic values, or at least two groups of existing characteristic values may be selected as the target existing characteristic values according to a preset rule, that is, at least two groups of target existing characteristic values in the target oil chromatogram data are obtained.
S402, inputting at least two groups of existing target characteristic values in the target oil chromatogram data into a trained abnormal data recognition model to obtain abnormal data in the at least two groups of existing target characteristic values.
The abnormal data identification model can be obtained by training through the model training method provided by any embodiment of the invention.
Optionally, the determined at least two groups of target existing characteristic values may be input into the trained abnormal data recognition model together, or the determined at least two groups of target existing characteristic values may be divided into multiple groups according to a certain division rule, and then input into the trained abnormal data recognition model respectively, that is, the at least two groups of target existing characteristic values in the target oil chromatogram data are input into the trained abnormal data recognition model.
Optionally, after the at least two groups of existing target characteristic values are input into the abnormal data identification model, the abnormal existing characteristic value data and the corresponding abnormal probability in the at least two groups of existing target characteristic values can be output, so that the abnormal data in the at least two groups of existing target characteristic values can be obtained, and the identification of the abnormal data in the oil chromatogram data can be completed.
Optionally, after outputting the abnormal existing characteristic value data and the corresponding abnormal probability, the characteristic value data with the abnormal probability value satisfying the preset condition can be directly discarded, so that the validity of the oil chromatogram data is ensured.
According to the scheme provided by the embodiment of the invention, at least two groups of target existing characteristic values in the target oil chromatographic data are obtained, the at least two groups of target existing characteristic values in the target oil chromatographic data are input into the trained abnormal data identification model to obtain the abnormal data in the at least two groups of target existing characteristic values, the trained abnormal data identification model can be utilized to effectively identify the abnormal data in the oil chromatographic data, the fault condition of the oil chromatographic acquisition device of the oil-immersed transformer is diagnosed in advance, the safe and stable operation of a power system is ensured, and the power supply reliability is improved.
EXAMPLE five
Fig. 5 is a block diagram of a model training apparatus according to a fifth embodiment of the present invention, where the model training apparatus according to the fifth embodiment of the present invention is capable of executing a model training method according to any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.
The model training apparatus may include: a determination module 501 and a training module 502.
The determining module 501 is configured to determine, according to existing characteristic values of at least two groups of samples in the sample oil chromatogram data, an extended characteristic value corresponding to the existing characteristic value of each group of samples; the existing characteristic values of different groups of samples are the content values of at least two gases in transformer oil acquired at different time points;
a training module 502, configured to train an abnormal data identification model according to the existing characteristic values of the at least two groups of samples, the extended characteristic values corresponding to the existing characteristic values of the at least two groups of samples, and the abnormal data labels associated with the sample oil chromatogram data, so as to identify abnormal data in the oil chromatogram data.
According to the scheme provided by the embodiment of the invention, the extended characteristic values are determined according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data, and then the abnormal data identification model is trained according to the existing characteristic values of at least two groups of samples, the corresponding extended characteristic values and the abnormal data labels associated with the sample oil chromatographic data, so that the model for identifying the abnormal data in the oil chromatographic data can be obtained.
Further, the determining module 501 may include:
the first determining unit is used for determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the correlation among the existing characteristic values of each group of samples in the sample oil chromatographic data;
and the second determining unit is used for determining a second expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the corresponding content value of the same gas in the existing characteristic values of different groups of samples.
Further, the first determining unit is specifically configured to:
analyzing the correlation between the existing characteristic values of each group of samples in the sample oil chromatographic data by adopting a rank sum check method to obtain at least two gas correlation values corresponding to the existing characteristic values of each group of samples;
determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to at least two gas correlation values corresponding to the existing characteristic values of each group of samples
Further, the second determining unit is specifically configured to:
determining the absolute growth amount and/or the relative growth rate of the target gas according to the corresponding content values of the target gas in the existing characteristic values of the first group of samples and the existing characteristic values of the second group of samples, and taking the absolute growth amount and/or the relative growth rate as a second expansion characteristic value corresponding to the existing characteristic values of the second group of samples;
wherein the first set of sample existing feature values are acquired at a time point earlier than the second set of sample existing feature values.
Further, the training module 502 is specifically configured to:
taking the existing characteristic value of each group of samples and the expanded characteristic value corresponding to the existing characteristic value of the group of samples as a group of data sets;
combining different groups of data sets according to the acquisition time of the existing characteristic values of each group of samples to obtain a training set, a verification set and a test set;
and training an abnormal data identification model based on the training set, the verification set and the test set and the abnormal data label associated with the sample oil chromatographic data.
Further, the abnormal data identification model is constructed on the basis of the long-short term memory LSTM network.
EXAMPLE six
Fig. 6 is a block diagram of a structure of an abnormal data identification apparatus based on oil chromatogram data according to a sixth embodiment of the present invention, and the abnormal data identification apparatus based on oil chromatogram data according to the sixth embodiment of the present invention can execute the abnormal data identification method based on oil chromatogram data according to any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.
The abnormal data identification device based on the oil chromatogram data can comprise an acquisition module 601 and a obtaining module 602.
The acquiring module 601 is configured to acquire at least two groups of existing target characteristic values in target oil chromatographic data, where the existing target characteristic values of different groups are content values of at least two gases in transformer oil collected at different time points;
an obtaining module 602, configured to input at least two groups of existing target characteristic values in the target oil chromatogram data into a trained abnormal data identification model, so as to obtain abnormal data in the at least two groups of existing target characteristic values;
the abnormal data identification model is obtained by training according to the model training method provided by any embodiment of the invention.
According to the scheme provided by the embodiment of the invention, at least two groups of target existing characteristic values in the target oil chromatographic data are obtained, the at least two groups of target existing characteristic values in the target oil chromatographic data are input into the trained abnormal data identification model to obtain the abnormal data in the at least two groups of target existing characteristic values, the trained abnormal data identification model can be utilized to effectively identify the abnormal data in the oil chromatographic data, the fault condition of the oil chromatographic acquisition device of the oil-immersed transformer is diagnosed in advance, the safe and stable operation of a power system is ensured, and the power supply reliability is improved.
EXAMPLE seven
Fig. 7 is a schematic structural diagram of an electronic device according to a seventh embodiment of the present invention. FIG. 7 illustrates a block diagram of an exemplary device suitable for use to implement embodiments of the present invention. The device shown in fig. 7 is only an example and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
As shown in FIG. 7, electronic device 12 is embodied in the form of a general purpose computing device. The components of electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
Electronic device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by electronic device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory (cache 32). The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, and commonly referred to as a "hard drive"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of embodiments described herein.
Electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with electronic device 12, and/or with any devices (e.g., network card, modem, etc.) that enable electronic device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the electronic device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be appreciated that although not shown in FIG. 7, other hardware and/or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, to name a few.
The processing unit 16 executes programs stored in the system memory 28 to perform various functional applications and data processing, such as implementing a model training method provided by an embodiment of the present invention and/or an abnormal data identification method based on oil chromatogram data.
Example eight
The eighth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program (or referred to as computer-executable instructions) is stored, where the computer program, when executed by a processor, is configured to perform the model training method provided in the embodiment of the present invention and/or the abnormal data identification method based on the oil chromatography data.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Python, Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the embodiments of the present invention have been described in more detail through the above embodiments, the embodiments of the present invention are not limited to the above embodiments, and many other equivalent embodiments may be included without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A method of model training, comprising:
determining an expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data; the existing characteristic values of different groups of samples are the content values of at least two gases in transformer oil collected at different time points;
and training an abnormal data identification model according to the at least two groups of existing characteristic values of the samples, the expanded characteristic values corresponding to the at least two groups of existing characteristic values of the samples and the abnormal data labels associated with the oil chromatogram data of the samples, wherein the abnormal data identification model is used for identifying abnormal data in the oil chromatogram data.
2. The method according to claim 1, wherein the determining the extended eigenvalue corresponding to the existing eigenvalue of each group according to the existing eigenvalue of at least two groups of samples in the sample oil chromatogram data comprises:
determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the correlation among the existing characteristic values of each group of samples in the sample oil chromatographic data;
and determining a second expansion characteristic value corresponding to the existing characteristic value of each group of samples according to the corresponding content value of the same gas in the existing characteristic values of different groups of samples.
3. The method according to claim 2, wherein the determining the first augmented eigenvalue corresponding to the existing eigenvalue of each group of samples according to the correlation between the existing eigenvalues of each group of samples in the sample oil chromatogram data comprises:
analyzing the correlation between the existing characteristic values of each group of samples in the sample oil chromatographic data by adopting a rank sum check method to obtain at least two gas correlation values corresponding to the existing characteristic values of each group of samples;
and determining a first expansion characteristic value corresponding to the existing characteristic value of each group of samples according to at least two gas correlation values corresponding to the existing characteristic values of each group of samples.
4. The method of claim 2, wherein determining the second augmented characteristic value corresponding to the existing characteristic value of each group of samples according to the corresponding content value of the same gas in the existing characteristic values of different groups of samples comprises:
determining the absolute growth amount and/or the relative growth rate of the target gas according to the corresponding content values of the target gas in the existing characteristic values of the first group of samples and the existing characteristic values of the second group of samples, and taking the absolute growth amount and/or the relative growth rate as a second expansion characteristic value corresponding to the existing characteristic values of the second group of samples;
wherein the first set of sample existing feature values are acquired at a time point earlier than the second set of sample existing feature values.
5. The method of claim 1, wherein training an abnormal data recognition model according to the existing eigenvalues of the at least two groups of samples, the expanded eigenvalues corresponding to the existing eigenvalues of the at least two groups of samples, and the abnormal data label associated with the sample oil chromatogram data comprises:
taking the existing characteristic value of each group of samples and the expanded characteristic value corresponding to the existing characteristic value of the group of samples as a group of data sets;
combining different groups of data sets according to the acquisition time of the existing characteristic values of each group of samples to obtain a training set, a verification set and a test set;
and training an abnormal data identification model based on the training set, the verification set and the test set and the abnormal data label associated with the sample oil chromatographic data.
6. The method of any of claims 1-5, wherein the anomaly data recognition model is constructed based on a long-short term memory (LSTM) network.
7. An abnormal data identification method based on oil chromatographic data is characterized by comprising the following steps:
acquiring at least two groups of existing target characteristic values in target oil chromatographic data, wherein the existing characteristic values of different groups of targets are content values of at least two gases in transformer oil collected at different time points;
inputting at least two groups of existing target characteristic values in the target oil chromatogram data into a trained abnormal data identification model to obtain abnormal data in the at least two groups of existing target characteristic values;
wherein the anomaly data recognition model is trained according to the method of any one of claims 1-6.
8. A model training apparatus, comprising:
the determining module is used for determining the expansion characteristic values corresponding to the existing characteristic values of each group of samples according to the existing characteristic values of at least two groups of samples in the sample oil chromatographic data; the existing characteristic values of different groups of samples are the content values of at least two gases in transformer oil collected at different time points;
and the training module is used for training an abnormal data identification model according to the existing characteristic values of the at least two groups of samples, the expanded characteristic values corresponding to the existing characteristic values of the at least two groups of samples and the abnormal data labels associated with the sample oil chromatographic data, and is used for identifying abnormal data in the oil chromatographic data.
9. An abnormal data recognition device based on oil chromatogram data, characterized by comprising:
the acquisition module is used for acquiring at least two groups of target existing characteristic values in target oil chromatographic data, wherein the different groups of target existing characteristic values are content values of at least two gases in transformer oil acquired at different time points;
the obtaining module is used for inputting at least two groups of target existing characteristic values in the target oil chromatographic data into a trained abnormal data identification model to obtain abnormal data in the at least two groups of target existing characteristic values;
wherein the anomaly data recognition model is trained according to the method of any one of claims 1-6.
10. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the model training method of any one of claims 1-6, and/or the method of oil chromatography data-based anomaly data identification of claim 7.
11. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out a model training method according to any one of claims 1 to 6 and/or an abnormal data identification method based on oil chromatogram data according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210399453.0A CN114818895A (en) | 2022-04-15 | 2022-04-15 | Model training and abnormal data identification method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210399453.0A CN114818895A (en) | 2022-04-15 | 2022-04-15 | Model training and abnormal data identification method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114818895A true CN114818895A (en) | 2022-07-29 |
Family
ID=82535869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210399453.0A Pending CN114818895A (en) | 2022-04-15 | 2022-04-15 | Model training and abnormal data identification method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114818895A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115236260A (en) * | 2022-09-16 | 2022-10-25 | 华谱科仪(北京)科技有限公司 | Chromatographic data storage method and device, electronic equipment and storage medium |
| CN116503411A (en) * | 2023-06-29 | 2023-07-28 | 博纯材料股份有限公司 | Chromatographic column state identification method and system based on image identification |
-
2022
- 2022-04-15 CN CN202210399453.0A patent/CN114818895A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115236260A (en) * | 2022-09-16 | 2022-10-25 | 华谱科仪(北京)科技有限公司 | Chromatographic data storage method and device, electronic equipment and storage medium |
| CN115236260B (en) * | 2022-09-16 | 2022-12-06 | 华谱科仪(北京)科技有限公司 | Chromatographic data storage method and device, electronic equipment and storage medium |
| CN116503411A (en) * | 2023-06-29 | 2023-07-28 | 博纯材料股份有限公司 | Chromatographic column state identification method and system based on image identification |
| CN116503411B (en) * | 2023-06-29 | 2023-08-29 | 博纯材料股份有限公司 | Chromatographic column state identification method and system based on image identification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114818895A (en) | Model training and abnormal data identification method, device, equipment and medium | |
| CN107908635B (en) | Method and device for establishing text classification model and text classification | |
| US9164878B2 (en) | Evaluating coverage of a software test | |
| CN110319982B (en) | Buried gas pipeline leakage judgment method based on machine learning | |
| CN113852603B (en) | Abnormality detection method and device for network traffic, electronic equipment and readable medium | |
| CN115032323A (en) | Method for determining oil chromatography data abnormity, electronic equipment and storage medium | |
| Lu et al. | Interval censored sampling plans for the gamma lifetime model | |
| CN114327077B (en) | Method and device for analyzing learner perception capability level based on eye movement tracking | |
| CN113033329A (en) | Method and device for judging abnormal answer of question in online education | |
| CN110209589B (en) | Knowledge base system test method, device, equipment and medium | |
| Van Petegem et al. | Pass/fail prediction in programming courses | |
| Chai et al. | Monitoring operational states of a nuclear reactor using seismoacoustic signatures and machine learning | |
| CN113919510A (en) | Sample feature selection method, device, equipment and medium | |
| CN113553245B (en) | Log anomaly detection method combining bidirectional slice GRU and gate control attention mechanism | |
| CN110263083B (en) | Knowledge graph processing method, device, equipment and medium | |
| CN117216558A (en) | Coal quality analysis model training method and device, electronic equipment and storage medium | |
| CN107328994A (en) | Insulation Resistance of Transformer experimental rig and method | |
| CN116521871A (en) | File detection method and device, processor and electronic equipment | |
| CN116720946A (en) | Credit risk prediction method, device and storage medium based on recurrent neural network | |
| CN113610344B (en) | Pollution early warning method and device, electronic equipment and storage medium | |
| CN110297989B (en) | Test method, device, equipment and medium for anomaly detection | |
| CN110427330B (en) | Code analysis method and related device | |
| Allam et al. | Assessment of statistical methodologies and pitfalls of dissertations carried out at National Cancer Institute, Cairo University | |
| CN113516398A (en) | Risk equipment identification method and device based on hierarchical sampling and electronic equipment | |
| CN113027432A (en) | Method, device, equipment and storage medium for determining total organic carbon content |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |