CN116823804B - Knowledge and data combined driving-based power transmission channel safety monitoring method - Google Patents
Knowledge and data combined driving-based power transmission channel safety monitoring method Download PDFInfo
- Publication number
- CN116823804B CN116823804B CN202310908405.4A CN202310908405A CN116823804B CN 116823804 B CN116823804 B CN 116823804B CN 202310908405 A CN202310908405 A CN 202310908405A CN 116823804 B CN116823804 B CN 116823804B
- Authority
- CN
- China
- Prior art keywords
- feature map
- feature
- color
- enhancement
- shallow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 100
- 238000000034 method Methods 0.000 title claims abstract description 75
- 238000012544 monitoring process Methods 0.000 title claims abstract description 53
- 238000001514 detection method Methods 0.000 claims abstract description 53
- 230000004927 fusion Effects 0.000 claims abstract description 41
- 239000003086 colorant Substances 0.000 claims abstract description 5
- 238000000605 extraction Methods 0.000 claims description 32
- 230000004913 activation Effects 0.000 claims description 26
- 238000011176 pooling Methods 0.000 claims description 22
- 238000001914 filtration Methods 0.000 claims description 11
- 230000007797 corrosion Effects 0.000 claims description 10
- 238000005260 corrosion Methods 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000011049 filling Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000003321 amplification Effects 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 3
- 238000005530 etching Methods 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims 1
- 238000010606 normalization Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 25
- 230000008569 process Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 6
- 239000000047 product Substances 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012633 leachable Substances 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the field of computer vision, in particular relates to a power transmission channel safety monitoring method based on knowledge and data combined driving, and aims to solve the problems of low safety monitoring precision and poor efficiency of a power transmission channel. The method comprises the following steps: acquiring a power transmission environment image and extracting a shallow feature map of the image; converting an input image into a gray image and extracting a contour feature map; performing contour feature enhancement and feature fusion operation on the shallow feature map and the contour feature map to obtain a contour enhanced feature map; extracting a color region feature map from an input image; performing color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color; weighting and fusing the shallow feature map, the contour enhancement feature map and the target enhancement feature map with specific colors; and replacing the shallow features in the shallow feature map, and performing target detection to obtain a power transmission channel safety monitoring result. The invention improves the safety monitoring precision and efficiency of the power transmission channel.
Description
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a power transmission channel safety monitoring method, system and device based on knowledge and data combined driving.
Background
The potential safety hazard targets influencing the normal operation of the transmission line in the transmission line channel and the surrounding environment can be detected, so that the safe and stable operation of power transmission can be ensured. Because the land of our country is wide, the topography is complex, the climate is changeable, with the continuous development of hardware equipment, the manual power safety monitoring mode that wastes time and energy is abandoned gradually, new line detection modes are emerging constantly, such as helicopter detection, robot detection, fixed embedded end detection, etc. The introduction of the new technology also brings new problems, a large amount of image data can be generated by utilizing a machine to collect the environment of the transmission channel, and the hidden danger position is screened from the massive data only by manpower, so that the hidden danger position is slow in speed and easy to miss. The problems can be effectively solved by introducing a target detection algorithm in computer vision, the model can replace manual work by learning the distribution rule in the potential safety hazard data, the power transmission channel data acquired by the automatic analysis machine can be judged with extremely high speed and higher accuracy, technicians are timely notified, and the occurrence of serious and oversized power accidents is suppressed. The potential safety hazard target detection of the power transmission channel is efficient and accurate, is not only a target pursued in academia, but also a difficult problem which needs to be solved urgently in the industry.
The current target detection algorithm based on deep learning mainly comprises a single-stage algorithm and a two-stage algorithm. Compared with a single-stage algorithm, the two-stage algorithm has the advantages that the step of extracting candidate frames is added, the possible positions of the targets are obtained in advance, the running speed is slower, and the speed is required for the safety monitoring of the transmission line channel, so that the single-stage algorithm is adopted. The single-stage algorithm comprises SSD and YOLO series, and due to the problems of engineering application and hardware deployment, the basic network uses YOLOv5, and the lattice point detection algorithm unifies feature extraction, identification and positioning in one network stage, so that the detection speed reaches real-time detection and simultaneously high-level detection effect can be maintained.
The traditional knowledge driving detection mode is very difficult to find the characteristic rule, and the characteristic cannot be manually extracted to summarize the target rule under different scenes, so that the traditional knowledge driving detection mode gradually goes to lonely. In recent years, a deep learning object detection mode driven by data only is raised, and the representation form of the feature can be automatically acquired by means of the data, so that the cost for acquiring the feature is greatly reduced, however, a large number of images are usually required to learn the feature of the object by means of the data drive, and in certain specific fields, enough images cannot be acquired to support model training. In the field of safety monitoring of power transmission channels, the problem of insufficient data is faced throughout the year, hidden danger of a circuit is a very small probability event, all possible hidden danger is difficult to be included in pictures collected by a machine, and once accidents occur, large-area power failure is caused, so that economic loss is difficult to measure. According to the invention, the advantages and the disadvantages of knowledge driving and data driving are comprehensively considered, and the current difficulties of power transmission channel safety monitoring are comprehensively considered, so that the power transmission channel safety monitoring method based on knowledge and data combined driving is provided, and the method is beneficial to more accurately detecting potential safety hazard targets on a small sample data set.
The prior art has the following problems: the detection method based on knowledge driving is difficult in manual feature design, time-consuming and labor-consuming, and targets in various scenes cannot be summarized. The detection method based on data driving can automatically learn the identification area, construct target characteristics and learn target rules from data under different environments, but the detection method based on data driving needs a large amount of data supporting learning and has unexplainability. The problem of insufficient data exists in the hidden danger of the transmission channel detected by using a data-driven detection algorithm YOLOv5, and the combined knowledge driving is needed to serve as guidance and supplement of information so as to obtain higher detection precision.
Disclosure of Invention
In order to solve the problems in the prior art, namely, the problems that characteristics cannot be summarized accurately and the model training cannot be supported by data volume only by means of knowledge or data driving detection mode, so that safety monitoring precision of a power transmission channel is low and efficiency is poor are solved, the first aspect of the invention provides a power transmission channel safety monitoring method based on knowledge and data combined driving, which comprises the following steps:
step S10, acquiring a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
Step S20, filtering and RGB converting are carried out on the input image to obtain a gray image, and a contour feature map of the gray image is extracted; the profile feature graphs are in one-to-one correspondence with target positions in the shallow feature graphs;
step S30, performing contour feature enhancement and feature fusion operation on the shallow feature map and the contour feature map to obtain a contour enhanced feature map;
step S40, extracting a color region feature map from the input image according to the set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
step S50, performing color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
step S60, combining the learner weights, and carrying out weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map to obtain data and knowledge joint enhancement features;
and step S70, replacing the shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further performing target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image.
In some preferred embodiments, filtering and RGB converting are performed on the input image to obtain a gray image, and a contour feature map of the gray image is extracted, where the method includes:
step S21, gaussian filtering is carried out on the input image, and an image after noise reduction is obtained:
I’ i ＝W G ΘI i ，i＝0,1,2,...
W G ＝[[1,2,1],[2,4,2],[1,2,1]]/16
wherein I represents the ith image, I i Representing the input image, I' i For Gaussian filtered output image, Θ represents convolutional filter operation, W G Is Gaussian convolution kernelParameters;
step S22, converting the noise-reduced image into a gray-scale image:
G i ＝0.299×I’ iR +0.587×I’ iG +0.114×I’ iB ，i＝0,1,2,...
wherein I' iR Red channel representing gaussian filtered output image, I' iG Green channel representing Gaussian filtered output image, I' iB Blue channel, G, representing Gaussian filtered output image i Representing a gray scale image;
step S23, calculating the profile features in the horizontal direction and the vertical direction by using a Sobel operator:
L iX ＝W SX ΘG i ，i＝0,1,2,...
L iY ＝W SY ΘG i ，i＝0,1,2,...
W SX ＝[[-1,0,1],[-2,0,2],[-1,0,1]]
W SY ＝[[1,2,1],[0,0,0],[-1,-2,-1]]
wherein W is SX Calculating a horizontal convolution kernel parameter, W, for a Sobel operator SY Calculating a vertical convolution kernel parameter for a Sobel operator, L iX Is a contour feature in the horizontal direction, L iY Is a vertical profile feature;
step S24, combining the profile features in the horizontal direction and the vertical direction:
L iS ＝|L iSX |+|L iSY |，i＝0,1,2,...
wherein L is iS And extracting overall contour features, namely a contour feature map, for the Sobel operator.
In some preferred embodiments, the contour feature enhancement and feature fusion operations are performed on the shallow feature map and the contour feature map, so as to obtain a contour enhanced feature map, which includes:
step S31, performing feature extraction and downsampling on the profile feature map to obtain a size-aligned profile feature:
L' iS ＝Conv u {L iS |u,k,p,s}，i＝0,1,2,...
wherein Conv u Represents a downsampling convolution operation, u represents a scaling multiple, k represents a convolution kernel size, p represents an edge filling number, s represents a convolution step size, L' iS Representing a size-aligned profile feature;
step S32, carrying out standardization and activation operation on the size alignment outline characteristics;
step S33, performing dimension amplification treatment on the dimension alignment outline characteristics after the standardization and activation operation:
wherein Conv d Representing a dimension change convolution operation, d representing the changed dimension,representing the size alignment profile features after the dimension is amplified;
step S34, performing hadamard product on the shallow features in the shallow feature map and the dimension alignment profile features after the dimension expansion, to obtain enhanced profile features:
wherein Hardmad represents Hadamard product operation, F i d The features of the shallow layer are represented by,representing enhanced profile features;
step S35, splicing the shallow layer features and the enhancement profile features, and carrying out feature extraction by convolution to obtain a mixed profile enhancement feature:
And S36, carrying out standardization and activation operation on the mixed contour enhancement features to obtain a contour enhancement feature map.
In some preferred embodiments, the set target color interval is obtained by the following method:
intercepting a hidden danger area containing set colors from a training image as a target hidden danger area, wherein the target hidden danger area contains a black part; the set color comprises red and yellow;
converting the target hidden danger area into HSV space;
and clustering the hue and saturation values of the target hidden danger area converted into the HSV space by using a two-dimensional kmeans algorithm to obtain hue and saturation value ranges:
wherein KMeans represents clustering operation, H is hue, S is saturation, w represents clustering dimension, n represents output cluster number,representing a clustering result;
wherein m is the number of values of hue and saturation,mean, sigma standard deviation, ++>As an upper limit of the number of the holes,for the lower limit->Representing the optimized clustering range, namely the hue and saturation value range;
calculating the maximum value and the minimum value in the cluster, and obtaining the light intensity numerical range:
wherein V is n D is a light intensity value set corresponding to the clustering interval n For the cluster interval sum V n Is a total collection of (1);
calculating the maximum value V nmax ：
V nmax ＝Max{V ni |V ni ∈V n }，i＝0,1,2,...
Where i is the ith intensity;
calculating the minimum value V nmin ：
V nmin ＝Min{V ni |V ni ∈V n }，i＝0,1,2,...
Calculating the total range:
V n ’＝[V nmin ,V nmax ]
wherein V is n ' is the total range of light intensity, HSV n Is the overall color interval, i.e., the set target color interval.
In some preferred embodiments, the color region feature map is extracted from the input image according to a set target color interval, and the method includes:
step S41, converting the input image from RGB space to HSV space;
step S42, extracting a corresponding color region from the input image converted into HSV space according to the set target color region;
wherein I is iHSV I is the HSV image, k is red, yellow and black, HSV k Representing the HSV color region range corresponding to k, inRange is the extraction corresponding region,for the extracted color region;
step S43, merging the color areas of the different color clusters to obtain the overall color cluster area characteristics:
wherein,is characterized by red area>Is characterized by yellow area>Is characterized by black area->Is a global color cluster region feature;
step S44, performing expansion operation on the overall color cluster region characteristics:
wherein, (M) 3,3 Is an expansion structural element with a core of 3, Is the expansion operation result;
step S45, performing corrosion operation on the overall color cluster region characteristics after expansion operation:
wherein, (N) 3,3 As an etching structural element with a core of 3,is the corrosion operation result;
step S46, performing an and operation on the input image and the general color cluster region characteristics after the corrosion operation:
wherein I is iRGB For an input image in RGB space, and represents And operates, F iHSV Is a color region feature map.
In some preferred embodiments, the method for obtaining the target enhancement feature map with specific color includes:
step S51, performing feature extraction and downsampling on the color region feature map to obtain a size-aligned color feature:
wherein Conv u Represents a downsampling convolution operation, u represents a scaling multiple, k represents a convolution kernel size, p represents an edge fill number, s represents a convolution step size,representing a size-aligned color feature, the number of channels being 3;
step S52, carrying out standardization and activation operation on the size alignment color features;
step S53, using maximum pooling and average pooling on the channel dimension for the normalized, activated, size-aligned color features:
Wherein d M ,d A For the output dimension, maxpool is max pooled, avgpool is average pooled,for maximum pooling results, < >>The result is averaged pooling;
step S54, splicing the maximum pooling result and the average pooling result, and adopting convolution dimension reduction to obtain dimension reduction color characteristics:
where v is the convolution kernel size, d C Concat is a feature splice for output dimension, conv v In order to reduce the dimension and extract the convolution of the features,is a dimension-reducing color feature;
step S55, performing an activation operation on the dimension-reduced color feature, to obtain a color region enhancement feature:
wherein,enhancing features for color regions;
step S56, performing matrix multiplication on the shallow features in the shallow feature map and the color region enhancement features, to obtain enhancement color region features:
wherein F is i d As a feature of the shallow layer,to enhance color region characteristics;
step S57, splicing the shallow layer features and the enhanced color region features, and carrying out feature extraction by convolution to obtain mixed color enhanced features;
and step S58, carrying out standardization and activation operation on the mixed color enhancement features to obtain a specific color target enhancement feature map.
In some preferred embodiments, the shallow feature map, the contour enhancement feature map, and the specific color target enhancement feature map are weighted and fused in combination with a learnable weight to obtain data and knowledge joint enhancement features, and the method comprises the following steps:
W＝(W 1 ,W 2 ,W 3 )
Wherein w is 1 ,w 2 ,w 3 Initial values are all 1, W represents a learnable weight, E is 0.00001, F i d As a feature of the shallow layer,for contour enhancement features in the contour enhancement feature map, < ->Enhancement features for specific color targets in the specific color target enhancement feature map, F i Is a hybrid fusion feature;
and performing activation operation and convolution operation on the mixed fusion characteristics to obtain data and knowledge joint enhancement characteristics.
In a second aspect of the present invention, a power transmission channel safety monitoring system based on knowledge and data combined driving is provided, the system comprising:
the input module is configured to acquire a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
the contour feature extraction module is configured to filter and RGB-convert the input image to obtain a gray image, and extract a contour feature map of the gray image; the profile feature graphs are in one-to-one correspondence with target positions in the shallow feature graphs;
the contour feature enhancement module is configured to perform contour feature enhancement and feature fusion operations on the shallow feature map and the contour feature map to obtain a contour enhancement feature map;
A color region feature extraction module configured to extract a color region feature map from the input image according to a set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
the color feature enhancement module is configured to perform color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
the weighted feature fusion module is configured to combine the learnable weights to perform weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map, so as to obtain data and knowledge combined enhancement features;
and the output module is configured to replace the shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further perform target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image.
In a third aspect of the present invention, a storage device is provided, in which a plurality of programs are stored, the programs being adapted to be loaded and executed by a processor to implement the above-described power transmission channel safety monitoring method based on knowledge and data combined driving.
In a fourth aspect of the present invention, a processing device is provided, including a processor and a storage device; the processor is suitable for executing each program; the storage device is suitable for storing a plurality of programs; the program is suitable for being loaded and executed by a processor to realize the power transmission channel safety monitoring method based on knowledge and data combined driving.
The invention has the beneficial effects that:
the invention improves the safety monitoring precision and efficiency of the power transmission channel.
(1) The method has strong characteristic expression capability and good generalization, can alleviate the problem that the data quantity is insufficient for supporting model training, has high processing speed, and accurately obtains hidden danger detection results of the power transmission channel;
(2) According to the method, a knowledge acquisition method is adopted to acquire the outline feature map of the hidden danger of the power transmission channel, outline feature enhancement and feature fusion operation are carried out on the shallow feature map and the outline feature map, the feature map which is guided by outline knowledge and has more generalization and interpretation can be acquired, and the dependency of a model on data is reduced by introducing key information to assist detection;
(3) According to the method, a knowledge acquisition method is adopted to acquire the appointed color characteristic region diagram of the hidden danger of the power transmission channel, color characteristic region enhancement and characteristic fusion operation are carried out on the shallow characteristic diagram and the color characteristic region diagram, more focused useful information and a more interpretable characteristic diagram guided by color knowledge can be acquired, the model accuracy is improved, the problem that the hidden danger data quantity of the power transmission channel is insufficient and the problem that the data driving generalization is poor can be solved;
(4) According to the method, the characteristic fusion proportion of the shallow characteristic map, the contour enhancement characteristic map and the specific color target enhancement characteristic map is controlled by utilizing the self-learning coefficient to carry out weighted fusion, so that the advantages of the characteristics are complemented, the characteristic duty ratio is automatically adjusted according to the importance of different characteristics, the optimal detection effect is achieved, and the precision and the efficiency of safety monitoring of a power transmission channel are improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings, in which:
FIG. 1 is a flow diagram of a knowledge-based and data-based joint drive power transmission channel security monitoring method in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of the overall structure of a power transmission channel safety monitoring method based on knowledge and data combined driving according to an embodiment of the present invention;
FIG. 3 is an exemplary diagram of shallow features of one embodiment of the invention;
FIG. 4 is a schematic diagram of a profile feature extraction module according to one embodiment of the invention;
FIG. 5 is a schematic diagram of a profile enhancement module of one embodiment of the present invention;
FIG. 6 is a diagram illustrating target color interval selection according to one embodiment of the present invention;
FIG. 7 is a schematic diagram of a color region feature extraction module according to one embodiment of the invention;
FIG. 8 is a schematic diagram of a color feature enhancement module of one embodiment of the present invention;
FIG. 9 is a schematic diagram of a weighted feature fusion module according to one embodiment of the invention;
FIG. 10 is a diagram illustrating input images, contour features, color region features, and object detection results according to an embodiment of the present invention.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
The invention discloses a power transmission channel safety monitoring method based on knowledge and data combined driving, which comprises the following steps:
step S10, acquiring a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
Step S20, filtering and RGB converting are carried out on the input image to obtain a gray image, and a contour feature map of the gray image is extracted; the profile feature graphs are in one-to-one correspondence with target positions in the shallow feature graphs;
step S30, performing contour feature enhancement and feature fusion operation on the shallow feature map and the contour feature map to obtain a contour enhanced feature map;
step S40, extracting a color region feature map from the input image according to the set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
step S50, performing color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
step S60, combining the learner weights, and carrying out weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map to obtain data and knowledge joint enhancement features;
and step S70, replacing the shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further performing target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image.
In order to more clearly describe the power transmission channel safety monitoring method based on knowledge and data combined driving of the present invention, each step in the method embodiment of the present invention is described in detail below with reference to fig. 1.
The power transmission channel safety monitoring method based on knowledge and data combined driving in one embodiment of the invention comprises the steps S10-S80, wherein the whole structure diagram is shown in fig. 2, and the steps are described in detail as follows:
step S10, acquiring a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
the method comprises the steps of firstly obtaining a power transmission environment image to be subjected to power transmission channel safety monitoring, wherein the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel, taking the power transmission environment image as an input image, and the size of the input image is 640 multiplied by 640; then, a shallow feature map of the input image is obtained through a pre-constructed target detection model (the data driving algorithm YOLOv5 is preferred in the invention); the shallow feature map is a structure diagram of the shallow feature map extraction process in the embodiment of the invention, wherein the resolution of the input image is reduced to 160×160 after two CBS passes.
Step S20, filtering and RGB converting are carried out on the input image to obtain a gray image, and a contour feature map of the gray image is extracted; the contour feature map corresponds to the target positions in the shallow feature map one by one.
Considering that the hidden trouble of the transmission line is mostly a crane, a truck, a tower crane and the like, the method has fixed and obvious profile characteristics, and the method can acquire the profile characteristics by adopting the traditional method, so that the calculation amount can be reduced, and meanwhile, the method can acquire good key characteristic information. Because noise interference exists in the original image, better contour features can be obtained by adding filtering operation, the image is converted into a gray image from RGB after filtering, the interference of a light source is reduced, and the contour features are conveniently extracted. The contour feature extraction method adopts a Sobel operator, and uses absolute value addition to approximate the square root opening number instead of the square root opening number, so that the calculation speed is improved. Fig. 4 shows an input image contour feature extraction module according to an embodiment of the invention. The specific process is as follows:
step S21, gaussian filtering is carried out on the input image, and an image after noise reduction is obtained:
I’ i ＝W G ΘI i ；W G ＝[[1,2,1],[2,4,2],[1,2,1]]/16
wherein I represents the ith image, I i Representing an input image, I i ' is a Gaussian filtered output image, Θ denotes a convolution filter operation, W G Is a gaussian convolution kernel parameter;
step S22, converting the noise-reduced image into a gray-scale image:
G i ＝0.299×I’ iR +0.587×I’ iG +0.114×I’ iB
wherein I' iR Red channel representing gaussian filtered output image, I' iG Green channel representing Gaussian filtered output image, I' iB Blue channel, G, representing Gaussian filtered output image i Representing a gray scale image;
step S23, calculating the profile features in the horizontal direction and the vertical direction by using a Sobel operator:
L iX ＝W SX ΘG i ；L iY ＝W SY ΘG i
W SX ＝[[-1,0,1],[-2,0,2],[-1,0,1]]
W SY ＝[[1,2,1],[0,0,0],[-1,-2,-1]]
wherein W is SX Calculating a horizontal convolution kernel parameter, W, for a Sobel operator SY Calculating a vertical convolution kernel parameter for a Sobel operator, L iX Is a contour feature in the horizontal direction, L iY Is a vertical profile feature;
step S24, combining the profile features in the horizontal direction and the vertical direction:
L iS ＝|L iSX |+|L iSY |，i＝0,1,2,...
wherein L is iS Extracting overall contour features, namely a contour feature map, for a Sobel operator;
through the operation, the contour knowledge extraction of the image is introduced into the deep learning algorithm, the contour feature extraction is synchronously carried out according to the input image in the model training, verification and test, the parameter is hardly introduced, and the method has high calculation speed and good contour information extraction capability.
Step S30, performing contour feature enhancement and feature fusion operation on the shallow feature map and the contour feature map to obtain a contour enhanced feature map;
The shallow characteristic size in the shallow characteristic diagram is 160×160, the input contour characteristic size is 640×640, the contour characteristic needs to be downsampled, and because of more contour image details and contour uncertainty caused by pooling, the method preferably adopts two convolution kernels of 3×3, the step length is 2, convolution self-adaptive contour local information with the dimension of 1 is output, the contour characteristic size is reduced from 640×640 to 160×160, and the characteristic size alignment is realized. The invention utilizes Hadamard product to control the amount of introduced information by enhancing the outline features in the shallow features, superimposes the enhanced features with the shallow features, and then performs feature characterization and fusion by convolution with a kernel of 3×3, as shown in FIG. 5. The method comprises the following steps:
step S31, performing feature extraction and downsampling on the profile feature map to obtain a size-aligned profile feature: ,
L' iS ＝Conv u {L iS |m,k,p,s}，i＝0,1,2,...
wherein Conv u Represents a downsampling convolution operation, u represents a scaling multiple, k represents a convolution kernel size, p represents an edge filling number, s represents a convolution step size, L' iS Representing a size-aligned profile feature; in the present invention, it is preferred to use Conv twice here u U is 1/2, k is 3, p is 1, s is 2;
Step S32, carrying out standardization and activation operation on the size alignment outline characteristics;
the invention adopts BN in standardization and SiLU in activation;
step S33, performing dimension amplification treatment on the dimension alignment outline characteristics after the standardization and activation operation:
wherein Conv d Representing a dimension change convolution operation, d representing the changed dimension,representing the size alignment profile features after the dimension is amplified; where d is preferably set to 64, k is 1, p is 0, s is 1 in the present invention;
step S34, performing hadamard product on the shallow features in the shallow feature map and the dimension alignment profile features after the dimension expansion, to obtain enhanced profile features:
wherein Hardmad represents Hadamard product operation, F i d The features of the shallow layer are represented by,representing enhanced profile features;
step S35, splicing the shallow layer features and the enhancement profile features, and carrying out feature extraction by convolution to obtain mixed profile enhancement features; the invention preferably adopts a kernel of 3, a convolution step length of 1, an edge filling quantity of 1 and an output of 64-dimensional convolution Conv g Mixing the features;
step S36, carrying out standardization and activation operation on the mixed contour enhancement features to obtain a contour enhancement feature map; the standardized method adopts BN preferably, and adopts SiLU for activation;
Through the operation, the method and the device enhance the shallow data characteristics by utilizing the profile knowledge, and enhance the key characteristics, so that the model has better generalization and interpretation, the detection effect is improved, and the problem of insufficient data is solved.
Step S40, extracting a color region feature map from the input image according to the set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
the method comprises the steps of carrying out tone and saturation clustering on an image converted into the HSV space, limiting a clustering range by removing abnormal values from a clustering result, obtaining the maximum value and the minimum value of the light intensity in the clustering result, obtaining a final HSV clustering result, and selecting a set target color interval, preferably comprising red, yellow and black, as shown in fig. 6. The target color interval acquisition process is as follows:
Intercepting a hidden danger area containing set colors from a training image as a target hidden danger area, wherein the target hidden danger area contains a black part;
because the shadow part of the hidden trouble of the power transmission line and the tyre part of the construction machinery are black, the black does not need to be taken out of the area separately;
converting the target hidden danger area into HSV space;
clustering the hue and saturation values of the target hidden danger area converted into the HSV space by using a two-dimensional kmeans algorithm to obtain hue and saturation value ranges;
the method comprises the following steps of:
two-dimensional kmeans clustering:
wherein KMeans represents clustering operation, H is hue, S is saturation, w represents clustering dimension, n represents output cluster number,representing a clustering result; in the present invention, w is preferably set to 2, and n has a maximum value of 12;
optimizing a clustering range:
wherein m is the number of values of hue and saturation,mean, sigma standard deviation, ++>As an upper limit of the number of the holes,for the lower limit->Representing the optimized cluster range, i.e. the hue and saturation value range.
Calculating the maximum value and the minimum value in the cluster for the light intensity value to obtain a light intensity value range;
The light intensity value range is obtained, and the calculation mode is as follows:
wherein V is n D is a light intensity value set corresponding to the clustering interval n For the cluster interval sum V n Is a total collection of (1);
calculating the maximum value V nmax ：
V nmax ＝Max{V ni |V ni ∈V n }，i＝0,1,2,...
Where i is the ith intensity;
calculating the minimum value V nmin ：
V nmin ＝Min{V ni |V ni ∈V n }，i＝0,1,2,...
Calculating the total range:
V n '＝[V nmin ,V nmax ]；
wherein V is n ' is the total range of light intensity, HSV n Is the overall color interval, i.e., the set target color interval.
The range interval obtained by the clustering method is more suitable for extracting the target area diagram of the hidden danger of the transmission channel, and the problem that targets cannot be effectively extracted by adopting the standard area is solved;
and then extracting a color region feature map from the input image according to the set target color interval, wherein the color region feature map is specifically as follows:
this section introduces color knowledge acquisition into the deep learning model with few parameters and faster computation speed, as well as good color region extraction capability. The input image is in an RGB format, and the input image needs to be converted into HSV space to facilitate region extraction. And extracting different color areas according to the multiple range intervals, and combining the acquired multiple color areas to obtain the overall color cluster area characteristics. Performing expansion operation on the overall color cluster region features, performing corrosion operation, reducing region holes, reducing information loss, and acquiring the color region features of the input image in an RGB format corresponding to the overall color cluster region features by using AND operation, as shown in FIG. 7, wherein the structure display diagram of the part is shown;
Step S41, converting the input image from RGB space to HSV space;
step S42, extracting a corresponding color region from the input image converted into HSV space according to the set target color interval:
wherein I is iHSV I is the number of images, k=r, Y, B is red, yellow, black, HSV k Representing the HSV color region range corresponding to k, inRange is the extraction corresponding region,for the extracted color region;
step S43, merging the color areas of the different color clusters to obtain the overall color cluster area characteristics:
wherein,is characterized by red area>Is characterized by yellow area>Is characterized by black area->Is a global color cluster region feature;
step S44, performing expansion operation on the overall color cluster region characteristics:
wherein, (M) 3,3 Is an expansion structural element with a core of 3,is the expansion operation result; in the present invention (M) 3,3 A rectangular structure is adopted;
step S45, performing corrosion operation on the overall color cluster region characteristics after expansion operation:
wherein, (N) 3,3 As an etching structural element with a core of 3,is the corrosion operation result; in the present invention (N) 3,3 A rectangular structure is adopted;
step S46, performing an and operation on the input image and the general color cluster region characteristics after the corrosion operation:
Wherein I is iRGB Inputting an image for RGB space, and representing And operating, F iHSV Is a color region feature map.
Through the operation, the color region characteristics of the target can be acquired by using a traditional method and transmitted to the next module, and color knowledge acquisition is introduced into an algorithm model in the section;
step S50, performing color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
the shallow characteristic size is 160 multiplied by 160, the size of an input color characteristic area diagram is 640 multiplied by 640, the color characteristic area diagram needs to be downsampled, and the problem of local detail information loss of the color area exists by pooling. And (3) carrying out channel maximum pooling and average pooling on the dimension-reduction color feature area diagram, splicing results, reducing dimension by convolution, processing the dimension-reduction result by using an activation function to increase nonlinearity, and obtaining color area enhancement features as area information to be focused. By matrix multiplication of the shallow features and the color features, the features of the target area acquire larger attention, and the duty ratio of useless information is reduced. The color region knowledge is introduced to make the detection process more interpretable, and a better detection effect is obtained under the condition of a small sample, and the structural diagram is shown in fig. 8, and is specifically as follows:
Step S51, performing feature extraction and downsampling on the color region feature map to obtain a size-aligned color feature:
wherein Conv u Represents a downsampling convolution operation, u represents a scaling multiple, k represents a convolution kernel size, p represents an edge fill number, s represents a convolution step size,representing a size-aligned color feature, the number of channels being 3;
the present invention uses twice Conv here u U is 1/2, k is 3, p is 1, s is 2;
step S52, carrying out standardization and activation operation on the size alignment color features;
the invention adopts BN in standardization and SiLU in activation;
step S53, using maximum pooling and average pooling on the channel dimension for the normalized, activated, size-aligned color features:
wherein d M ,d A For the output dimension, maxpool is max pooled, avgpool is average pooled,for maximum pooling results, < >>The result is averaged pooling;
d in the invention M ,d A Are all 1;
step S54, splicing the maximum pooling result and the average pooling result, and adopting convolution dimension reduction to obtain dimension reduction color characteristics:
where v is the convolution kernel size, d C Concat is a feature splice for output dimension, conv v In order to reduce the dimension and extract the convolution of the features,is a dimension-reducing color feature; v is 5,d C 1 is shown in the specification;
step S55, performing an activation operation on the dimension-reduced color feature, to obtain a color region enhancement feature:
wherein,enhancing features for color regions; />
Step S56, performing matrix multiplication on the shallow features in the shallow feature map and the color region enhancement features, to obtain enhancement color region features:
wherein F is i d As a feature of the shallow layer,to enhance color region characteristics;
step S57, splicing the shallow layer features and the enhanced color region features, and carrying out feature extraction by convolution to obtain mixed color enhanced features;
the method adopts a convolution Conv with a kernel of 3, a convolution step length of 1, an edge filling quantity of 1 and an output of 64 dimensions g Mixing the features;
step S58, carrying out standardization and activation operation on the mixed color enhancement features to obtain a specific color target enhancement feature map;
the invention adopts BN in standardization and SiLU in activation;
by the method, the color knowledge is utilized to guide the data driving algorithm, the human brain searching information is simulated, the target information is enhanced, the detection effect is improved, the key information is artificially enhanced, the data driving is assisted, and the problem of insufficient data is solved.
Step S60, combining the learner weights, and carrying out weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map to obtain data and knowledge joint enhancement features;
Because the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map have different advantages and disadvantages, the invention sets the self-learning coefficient, and the model automatically optimizes the duty ratio of each feature to achieve the best detection effect, and the structure diagram is shown in fig. 9, and specifically comprises the following steps:
step S61, setting a self-learning coefficient W;
W＝(W 1 ,W 2 ,W 3 )；
wherein w is 1 ,w 2 ,w 3 The initial values are all 1, W represents a leachable weight, E is 0.00001, and i represents an ith coefficient;
when the coefficient is negative, the characteristics of the same level are cancelled, so that the characteristics are lost, the absolute value is adopted to prevent the negative number, and the addition of E ensures that the denominator is not 0;
step S62, weighting and fusing the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map:
wherein F is i d As a feature of the shallow layer,for contour enhancement features in the contour enhancement feature map, < ->Enhancing a particular color target in a feature map for a particular color targetStandard enhancement features, F i Is a fusion feature;
step S73, performing an activating operation and a convolution operation on the mixed fusion characteristics;
the activation of the invention preferably uses SiLU, convolution Conv k The convolution kernel is 1, and the step length is 1;
the operation can adaptively adjust the duty ratio of each characteristic, fully combine the advantages and disadvantages of the characteristic, adjust the duty ratio of knowledge driving and data driving, and acquire better detection precision.
And step S70, replacing the shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further performing target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image.
Fig. 10 is a diagram showing the result of a power transmission channel safety monitoring method based on knowledge and data combined driving, wherein the first column is an input image, the second column is a contour feature, the third column is a color region feature, and the fourth column is a detection result.
The second embodiment of the invention provides a knowledge and data combined driving-based power transmission channel safety monitoring system, which comprises:
the input module is configured to acquire a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
the contour feature extraction module is configured to filter and RGB-convert the input image to obtain a gray image, and extract a contour feature map of the gray image; the profile feature graphs are in one-to-one correspondence with target positions in the shallow feature graphs;
The contour feature enhancement module is configured to perform contour feature enhancement and feature fusion operations on the shallow feature map and the contour feature map to obtain a contour enhancement feature map;
a color region feature extraction module configured to extract a color region feature map from the input image according to a set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
the color feature enhancement module is configured to perform color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
the weighted feature fusion module is configured to combine the learnable weights to perform weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map, so as to obtain data and knowledge combined enhancement features;
and the output module is configured to replace the shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further perform target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image.
It will be clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the system described above and the related description may refer to the corresponding process in the foregoing method embodiment, which is not repeated here.
It should be noted that, in the power transmission channel safety monitoring system based on knowledge and data combined driving provided in the foregoing embodiment, only the division of the foregoing functional modules is illustrated, in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the modules or steps in the foregoing embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present invention are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present invention.
A storage device according to a third embodiment of the present invention stores therein a plurality of programs adapted to be loaded and executed by a processor to implement the above-described knowledge-based and data-based joint driving power transmission channel safety monitoring method.
A processing device according to a fourth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute each program; a storage device adapted to store a plurality of programs; the program is suitable for being loaded and executed by a processor to realize the power transmission channel safety monitoring method based on knowledge and data combined driving.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the storage device and the processing device described above and the related description may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
Those of skill in the art will appreciate that the various illustrative modules, method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the program(s) corresponding to the software modules, method steps, may be embodied in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not intended to be limiting.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus/apparatus.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will fall within the scope of the present invention.
Claims (9)
1. The utility model provides a power transmission channel safety monitoring method based on knowledge and data combined drive which is characterized in that the method comprises the following steps:
step S10, acquiring a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
Step S20, filtering and RGB converting are carried out on the input image to obtain a gray image, and a contour feature map of the gray image is extracted; the profile feature graphs are in one-to-one correspondence with target positions in the shallow feature graphs;
step S30, performing contour feature enhancement and feature fusion operation on the shallow feature map and the contour feature map to obtain a contour enhanced feature map;
step S40, extracting a color region feature map from the input image according to the set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
step S50, performing color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
step S60, combining the learner weights, and carrying out weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map to obtain data and knowledge joint enhancement features;
step S70, replacing shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further performing target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image;
The set target color interval is obtained by the following steps:
intercepting a hidden danger area containing set colors from a training image as a target hidden danger area, wherein the target hidden danger area contains a black part; the set color comprises red and yellow;
converting the target hidden danger area into HSV space;
and clustering the hue and saturation values of the target hidden danger area converted into the HSV space by using a two-dimensional kmeans algorithm to obtain hue and saturation value ranges:
wherein KMeans represents clustering operation, H is hue, S is saturation, w represents clustering dimension, n represents output cluster number,representing a clustering result;
wherein m is the number of values of hue and saturation,mean, sigma standard deviation, ++>Is the upper limit of->For the lower limit->Representing the optimized clustering range, namely the hue and saturation value range;
calculating the maximum value and the minimum value in the cluster, and obtaining the light intensity numerical range:
wherein V is n D is a light intensity value set corresponding to the clustering interval n For the cluster interval sum V n Is a total collection of (1);
calculating the maximum value V nmax ：
V nmax ＝Max{V ni |V ni ∈V n }，i＝0,1,2,...
Where i is the ith intensity;
calculating the minimum value V nmin ：
V nmin ＝Min{V ni |V ni ∈V n }，i＝0,1,2,...
Calculating the total range:
V′ n ＝[V nmin ,V nmax ]
wherein V 'is' n HSV, the total range of light intensity n Is the overall color interval, i.e., the set target color interval.
2. The power transmission channel safety monitoring method based on knowledge and data combined driving according to claim 1, wherein filtering and RGB conversion are performed on the input image to obtain a gray image, and a contour feature map of the gray image is extracted, and the method comprises the following steps:
step S21, gaussian filtering is carried out on the input image, and an image after noise reduction is obtained:
I′ i ＝W G ΘI i ，i＝0,1,2,...
W G ＝[[1,2,1],[2,4,2],[1,2,1]]/16
wherein I represents the ith image, I i Representing the input image, I' i For Gaussian filtered output image, Θ represents convolutional filter operation, W G Is a gaussian convolution kernel parameter;
step S22, converting the noise-reduced image into a gray-scale image:
G i ＝0.299×I′ iR +0.587×I′ iG +0.114×I′ iB ，i＝0,1,2,...
wherein I' iR Red channel representing gaussian filtered output image, I' iG Green channel representing Gaussian filtered output image, I' iB Blue channel, G, representing Gaussian filtered output image i Representing a gray scale image;
step S23, calculating the profile features in the horizontal direction and the vertical direction by using a Sobel operator:
L iX ＝W SX ΘG i ，i＝0,1,2,...
L iY ＝W SY ΘG i ，i＝0,1,2,...
W SX ＝[[-1,0,1],[-2,0,2],[-1,0,1]]
W SY ＝[[1,2,1],[0,0,0],[-1,-2,-1]]
wherein W is SX Calculating a horizontal convolution kernel parameter, W, for a Sobel operator SY Calculating a vertical convolution kernel parameter for a Sobel operator, L iX Is a contour feature in the horizontal direction, L iY Is a vertical profile feature;
Step S24, combining the profile features in the horizontal direction and the vertical direction:
L iS ＝|L iX |+|L iY |，i＝0,1,2,...
wherein L is iS And extracting overall contour features, namely a contour feature map, for the Sobel operator.
3. The knowledge and data combined driving-based power transmission channel safety monitoring method according to claim 2, wherein contour feature enhancement and feature fusion operations are performed on the shallow feature map and the contour feature map, and a contour enhancement feature map is obtained, and the method comprises the following steps:
step S31, performing feature extraction and downsampling on the profile feature map to obtain a size-aligned profile feature:
L' iS ＝Conv u {L iS |u,k,p,s}，i＝0,1,2,...
wherein Conv u Represents a downsampling convolution operation, u represents a scaling multiple, k represents a convolution kernel size, p represents an edge filling number, s represents a convolution step size, L' iS Representing a size-aligned profile feature;
step S32, carrying out standardization and activation operation on the size alignment outline characteristics;
step S33, performing dimension amplification treatment on the dimension alignment outline characteristics after the standardization and activation operation:
wherein,Conv d representing a dimension change convolution operation, d representing the changed dimension,representing the size alignment profile features after the dimension is amplified;
step S34, performing hadamard product on the shallow features in the shallow feature map and the dimension alignment profile features after the dimension expansion, to obtain enhanced profile features:
Wherein Hardmad represents Hadamard product operation, F i d The features of the shallow layer are represented by,representing enhanced profile features;
step S35, splicing the shallow layer features and the enhancement profile features, and carrying out feature extraction by convolution to obtain a mixed profile enhancement feature:
and S36, carrying out standardization and activation operation on the mixed contour enhancement features to obtain a contour enhancement feature map.
4. The knowledge and data combined driving-based power transmission channel safety monitoring method according to claim 3, wherein a color region feature map is extracted from the input image according to a set target color interval, and the method comprises the following steps:
step S41, converting the input image from RGB space to HSV space;
step S42, extracting a corresponding color region from the input image converted into HSV space according to the set target color region;
wherein I is iHSV Is an HSV image, k=R, Y and B are red, yellow and black respectively, and HSV k Representing the HSV color region range corresponding to k, inRange is the extraction corresponding region,for the extracted color region;
step S43, merging the color areas of the different color clusters to obtain the overall color cluster area characteristics:
wherein,is characterized by red area >Is characterized by yellow area>Is characterized by a black area,is a global color cluster region feature;
step S44, performing expansion operation on the overall color cluster region characteristics:
wherein, (M) 3,3 Is an expansion structural element with a core of 3,is the expansion operation result;
step S45, performing corrosion operation on the overall color cluster region characteristics after expansion operation:
wherein, (N) 3,3 As an etching structural element with a core of 3,is the corrosion operation result;
step S46, performing an and operation on the input image and the general color cluster region characteristics after the corrosion operation:
wherein I is iRGB For an input image in RGB space, and represents And operates, F iHSV Is a color region feature map.
5. The knowledge and data combined driving-based power transmission channel safety monitoring method according to claim 4, wherein the shallow feature map and the color region feature map are subjected to color region feature enhancement and feature fusion operations to obtain a specific color target enhancement feature map, and the method comprises the following steps:
step S51, performing feature extraction and downsampling on the color region feature map to obtain a size-aligned color feature:
wherein Conv u Represents a downsampling convolution operation, u represents a scaling multiple, k represents a convolution kernel size, p represents an edge fill number, s represents a convolution step size, Representing a size-aligned color feature, the number of channels being 3;
step S52, performing a normalization and activation operation on the size-alignment color feature:
step S53, using maximum pooling and average pooling on the channel dimension for the normalized, activated, size-aligned color features:
wherein d M ,d A For the output dimension, maxpool is max pooled, avgpool is average pooled,for maximum pooling results, < >>The result is averaged pooling;
step S54, splicing the maximum pooling result and the average pooling result, and adopting convolution dimension reduction to obtain dimension reduction color characteristics:
where v is the convolution kernel size, d C Concat is a feature splice for output dimension, conv v In order to reduce the dimension and extract the convolution of the features,is a dimension-reducing color feature;
step S55, performing an activation operation on the dimension-reduced color feature, to obtain a color region enhancement feature:
wherein,enhancing features for color regions;
step S56, performing matrix multiplication on the shallow features in the shallow feature map and the color region enhancement features, to obtain enhancement color region features:
wherein F is i d As a feature of the shallow layer,to enhance color region characteristics;
step S57, splicing the shallow layer features and the enhanced color region features, and carrying out feature extraction by convolution to obtain mixed color enhanced features;
And step S58, carrying out standardization and activation operation on the mixed color enhancement features to obtain a specific color target enhancement feature map.
6. The power transmission channel safety monitoring method based on knowledge and data combined driving according to claim 5, wherein the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map are weighted and fused by combining a learnable weight, so as to obtain data and knowledge combined enhancement features, and the method comprises the following steps:
W＝(W 1 ,W 2 ,W 3 )
wherein w is 1 ,w 2 ,w 3 Initial values are all 1, W represents a learnable weight, E is 0.00001, F i d As a feature of the shallow layer,for contour enhancement features in the contour enhancement feature map, < ->Enhancement features for specific color targets in the specific color target enhancement feature map, F i Is a hybrid fusion feature;
and performing activation operation and convolution operation on the mixed fusion characteristics to obtain data and knowledge joint enhancement characteristics.
7. A knowledge and data combined drive-based power transmission channel safety monitoring system, comprising:
the input module is configured to acquire a power transmission environment image to be subjected to power transmission channel safety monitoring as an input image; extracting a shallow feature map of the input image through a pre-constructed target detection model; the power transmission environment image comprises a power transmission channel and surrounding environments of the power transmission channel;
The contour feature extraction module is configured to filter and RGB-convert the input image to obtain a gray image, and extract a contour feature map of the gray image; the profile feature graphs are in one-to-one correspondence with target positions in the shallow feature graphs;
the contour feature enhancement module is configured to perform contour feature enhancement and feature fusion operations on the shallow feature map and the contour feature map to obtain a contour enhancement feature map;
a color region feature extraction module configured to extract a color region feature map from the input image according to a set target color interval; the color region feature map corresponds to the target positions in the shallow feature map one by one;
the color feature enhancement module is configured to perform color region feature enhancement and feature fusion operation on the shallow feature map and the color region feature map to obtain a target enhancement feature map with a specific color;
the weighted feature fusion module is configured to combine the learnable weights to perform weighted fusion on the shallow feature map, the contour enhancement feature map and the specific color target enhancement feature map, so as to obtain data and knowledge combined enhancement features;
the output module is configured to replace shallow features in the shallow feature map with the data and knowledge combined enhancement features, and further perform target detection through the target detection model to obtain a power transmission channel safety monitoring result corresponding to the input image;
The set target color interval is obtained by the following steps:
intercepting a hidden danger area containing set colors from a training image as a target hidden danger area, wherein the target hidden danger area contains a black part; the set color comprises red and yellow;
converting the target hidden danger area into HSV space;
and clustering the hue and saturation values of the target hidden danger area converted into the HSV space by using a two-dimensional kmeans algorithm to obtain hue and saturation value ranges:
wherein KMeans represents clustering operation, H is hue, S is saturation, w represents clustering dimension, n represents output cluster number,representing a clustering result;
wherein m is the number of values of hue and saturation,mean, sigma standard deviation, ++>Is the upper limit of->For the lower limit->Representing the optimized clustering range, namely the hue and saturation value range;
calculating the maximum value and the minimum value in the cluster, and obtaining the light intensity numerical range:
wherein V is n D is a light intensity value set corresponding to the clustering interval n For the cluster interval sum V n Is a total collection of (1);
calculating the maximum value V nmax ：
V nmax ＝Max{V ni |V ni ∈V n }，i＝0,1,2,...
Where i is the ith intensity;
calculating the minimum value V nmin ：
V nmin ＝Min{V ni |V ni ∈V n }，i＝0,1,2,...
Calculating the total range:
V′ n ＝[V nmin ,V nmax ]
wherein V 'is' n HSV, the total range of light intensity n Is the overall color interval, i.e., the set target color interval.
8. A storage device in which a plurality of programs are stored, characterized in that the programs are adapted to be loaded and executed by a processor to implement the knowledge-based and data-based joint drive power transmission channel safety monitoring method according to any one of claims 1-6.
9. A processing apparatus includes
A processor adapted to execute each program; and
a storage device adapted to store a plurality of programs;
a power transmission channel safety monitoring method based on knowledge and data combination driving according to any one of claims 1-6, characterized in that the program is adapted to be loaded and executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310908405.4A CN116823804B (en) | 2023-07-21 | 2023-07-21 | Knowledge and data combined driving-based power transmission channel safety monitoring method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310908405.4A CN116823804B (en) | 2023-07-21 | 2023-07-21 | Knowledge and data combined driving-based power transmission channel safety monitoring method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116823804A CN116823804A (en) | 2023-09-29 |
| CN116823804B true CN116823804B (en) | 2024-02-09 |
Family
ID=88124133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310908405.4A Active CN116823804B (en) | 2023-07-21 | 2023-07-21 | Knowledge and data combined driving-based power transmission channel safety monitoring method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116823804B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018086299A1 (en) * | 2016-11-11 | 2018-05-17 | 广东电网有限责任公司清远供电局 | Image processing-based insulator defect detection method and system |
| CN111310899A (en) * | 2020-02-19 | 2020-06-19 | 山东大学 | Electric power defect identification method based on symbiotic relationship and small sample learning |
| CN111680706A (en) * | 2020-06-17 | 2020-09-18 | 南开大学 | Double-channel output contour detection method based on coding and decoding structure |
| CN112733610A (en) * | 2020-12-15 | 2021-04-30 | 国网辽宁省电力有限公司技能培训中心 | Power transmission line image recognition detection method based on unmanned aerial vehicle |
| CN113962353A (en) * | 2020-07-20 | 2022-01-21 | 陈永聪 | Method for establishing strong artificial intelligence |
| CN114255183A (en) * | 2021-12-17 | 2022-03-29 | 中国科学院上海光学精密机械研究所 | Intelligent computing optical imaging method driven by data and knowledge in combined mode |
| CN115761644A (en) * | 2022-11-25 | 2023-03-07 | 国网山东省电力公司超高压公司 | Transmission line foreign matter detection method based on deep learning and frame difference method |
| CN116310386A (en) * | 2023-03-16 | 2023-06-23 | 郑州轻工业大学 | Shallow adaptive enhanced context-based method for detecting small central Net target |
-
2023
- 2023-07-21 CN CN202310908405.4A patent/CN116823804B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018086299A1 (en) * | 2016-11-11 | 2018-05-17 | 广东电网有限责任公司清远供电局 | Image processing-based insulator defect detection method and system |
| CN111310899A (en) * | 2020-02-19 | 2020-06-19 | 山东大学 | Electric power defect identification method based on symbiotic relationship and small sample learning |
| CN111680706A (en) * | 2020-06-17 | 2020-09-18 | 南开大学 | Double-channel output contour detection method based on coding and decoding structure |
| CN113962353A (en) * | 2020-07-20 | 2022-01-21 | 陈永聪 | Method for establishing strong artificial intelligence |
| CN112733610A (en) * | 2020-12-15 | 2021-04-30 | 国网辽宁省电力有限公司技能培训中心 | Power transmission line image recognition detection method based on unmanned aerial vehicle |
| CN114255183A (en) * | 2021-12-17 | 2022-03-29 | 中国科学院上海光学精密机械研究所 | Intelligent computing optical imaging method driven by data and knowledge in combined mode |
| CN115761644A (en) * | 2022-11-25 | 2023-03-07 | 国网山东省电力公司超高压公司 | Transmission line foreign matter detection method based on deep learning and frame difference method |
| CN116310386A (en) * | 2023-03-16 | 2023-06-23 | 郑州轻工业大学 | Shallow adaptive enhanced context-based method for detecting small central Net target |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116823804A (en) | 2023-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111444821B (en) | Automatic identification method for urban road signs | |
| CN108009542B (en) | Weed image segmentation method in rape field environment | |
| CN105678278A (en) | Scene recognition method based on single-hidden-layer neural network | |
| CN114842208B (en) | Deep learning-based power grid harmful bird species target detection method | |
| CN110532961B (en) | Semantic traffic light detection method based on multi-scale attention mechanism network model | |
| CN112613454A (en) | Electric power infrastructure construction site violation identification method and system | |
| CN112784869B (en) | Fine-grained image identification method based on attention perception and counterstudy | |
| CN101561932A (en) | Method and device for detecting real-time movement target under dynamic and complicated background | |
| CN111563408B (en) | High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning | |
| CN114299036B (en) | Electronic element detection method and device, storage medium and electronic equipment | |
| CN114241326B (en) | Progressive intelligent production method and system for ground feature elements of remote sensing images | |
| CN114155468A (en) | Method, device, equipment and medium for detecting oil leakage of transformer | |
| CN116823804B (en) | Knowledge and data combined driving-based power transmission channel safety monitoring method | |
| CN113671599A (en) | Global climate mode-based login cyclone identification method | |
| CN113052103A (en) | Electrical equipment defect detection method and device based on neural network | |
| CN116884192A (en) | Power production operation risk early warning method, system and equipment | |
| CN109636194B (en) | Multi-source cooperative detection method and system for major change of power transmission and transformation project | |
| CN116721385A (en) | Machine learning-based RGB camera data cyanobacteria bloom monitoring method | |
| CN106682668A (en) | Power transmission line geological disaster monitoring method using unmanned aerial vehicle to mark images | |
| Doxani et al. | Automatic change detection in urban areas under a scale-space, object-oriented classification framework | |
| CN111259981B (en) | Automatic classification system after remote sensing image processing | |
| CN115019306A (en) | Embedding box label batch identification method and system based on deep learning and machine vision | |
| CN115294461B (en) | Power facility collapse and inclination assessment method and system based on BIM and remote sensing image | |
| CN113780439B (en) | Multi-cloud identification system of different types of meteorological satellites based on unsupervised domain adaptation | |
| CN116907450A (en) | Pollution discharge outlet pollutant tracing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |