CN117560511A - Spacer image compression method and system based on graph segmentation technology and electric power inspection - Google Patents
Spacer image compression method and system based on graph segmentation technology and electric power inspection Download PDFInfo
- Publication number
- CN117560511A CN117560511A CN202311631234.1A CN202311631234A CN117560511A CN 117560511 A CN117560511 A CN 117560511A CN 202311631234 A CN202311631234 A CN 202311631234A CN 117560511 A CN117560511 A CN 117560511A
- Authority
- CN
- China
- Prior art keywords
- image
- graph
- segmentation
- compression
- spacer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007906 compression Methods 0.000 title claims abstract description 80
- 230000006835 compression Effects 0.000 title claims abstract description 80
- 230000011218 segmentation Effects 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000007689 inspection Methods 0.000 title claims abstract description 70
- 125000006850 spacer group Chemical group 0.000 title claims abstract description 55
- 238000005516 engineering process Methods 0.000 title claims abstract description 50
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 31
- 230000007246 mechanism Effects 0.000 claims abstract description 24
- 238000012800 visualization Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000005192 partition Methods 0.000 claims description 15
- 230000005540 biological transmission Effects 0.000 claims description 14
- 238000013139 quantization Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000003709 image segmentation Methods 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 7
- 238000013144 data compression Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000010191 image analysis Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 229940060587 alpha e Drugs 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000003708 edge detection Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- 238000013210 evaluation model Methods 0.000 claims description 3
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 230000008521 reorganization Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000000007 visual effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a spacer image compression method and system based on a graph segmentation technology and used for power inspection, and relates to the technical field of power inspection. The method comprises the steps of collecting a spacer image obtained by inspection equipment, and carrying out segmentation processing on a background and a target area in the image; different compression rates are respectively set for a target area and a background area in an image, and coding is carried out based on a JPEG compression technology; storing and transmitting the coded data to a terminal; the terminal decodes the encoded data based on the JPEG compression technology to restore the compressed image; super-division enhancement is carried out on the image by a SwinIR algorithm based on a multi-level attention mechanism; an enhanced high-visualization compressed image is obtained. The invention has the beneficial effects that the compression and processing operation is more efficient, and the computing resource and the time cost are saved.
Description
Technical Field
The invention relates to the technical field of power inspection, in particular to a spacer image compression method based on a graph segmentation technology for power inspection.
Background
In the power line inspection, the spacer has the functions of fixing and supporting wires, safety isolation, convenience in maintenance and operation and the like, and the state and the safety of the spacer play a vital role in the normal operation of a power system. The inspection of the existing power equipment spacer is usually carried out manually, and the problems of manpower and material resource waste, low working efficiency, safety risk and the like exist in the mode. In recent years, with the rapid development of automation technology, the inspection of the spacer by using inspection equipment has become a potential solution. The inspection equipment can improve inspection efficiency, reduce personnel risks and realize comprehensive monitoring of large-area and high-difficulty areas. However, the inspection equipment can acquire a large amount of image data, and the conventional image compression method cannot meet the dual requirements of image quality and data transmission efficiency. Therefore, it is necessary to compress and process the images in different proportions according to the data with different importance levels in the images, so as to improve the image compression efficiency, reduce the data transmission amount, and ensure the visual quality of the compressed images.
Some image compression methods exist at present, and have problems of information loss and low compression ratio when processing high-resolution images. The inspection device spacer inspection image is characterized in that an obvious segmentation boundary exists between the background and the target.
Disclosure of Invention
This section is intended to outline some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section as well as in the description summary and in the title of the application, to avoid obscuring the purpose of this section, the description summary and the title of the invention, which should not be used to limit the scope of the invention.
The present invention has been made in view of the above-mentioned or existing problems occurring in the prior art.
Therefore, the invention aims to provide a spacer image compression method based on the image segmentation technology and the power inspection, which can effectively distinguish front and rear scenic areas of an image, combine the importance of different areas in the image to perform lossy or lossless compression, realize the efficient compression of the image on the premise of meeting the image quality requirement and reduce the transmission quantity and storage space of image data.
In order to solve the technical problems, the present invention provides the following technical solutions: a spacer image compression method based on image segmentation technology and electric power inspection comprises the steps of 1, collecting a spacer image obtained by inspection equipment, and carrying out segmentation processing on a background and a target area in the image;
step 2, respectively setting different compression rates for a target area and a background area in an image, and encoding based on a JPEG compression technology;
step 3, storing JPEG encoded data and transmitting the JPEG encoded data to a terminal;
step 4, decoding the coded data by the terminal based on the JPEG technology to restore the compressed image;
step 5, performing super-division enhancement on the image by a SwinIR algorithm based on a multi-level attention mechanism;
and 6, obtaining the enhanced high-visualization compressed image.
As a preferable scheme of the spacer image compression method based on the graph segmentation technology power inspection, the invention comprises the following steps: the step 1 also comprises preprocessing the acquired spacer images, including adjusting the size, marking targets, and amplifying image data in a rotating and overturning mode, wherein the data set is divided into a training set and a verification set.
As a preferable scheme of the spacer image compression method based on the graph segmentation technology power inspection, the invention comprises the following steps: in the step 1, a segmentation technology based on feature cascade and an attention mechanism carries out segmentation processing on a target area and a background area; the method specifically comprises the following steps:
step 1.1, dividing an input image into a plurality of connected uniform superpixels by using an SLIC superpixel generation algorithm, reducing the complexity of subsequent image processing, and retaining main structural information of the image;
step 1.2, performing feature cascade analysis, and calculating the color, texture and shape features of each super pixel; calculating the average value of all pixel colors in the super pixel to obtain a vector representing the color; calculating texture features of pixels within the superpixel using the texture descriptors; obtaining the boundary information of the super pixel by using a boundary contour extraction algorithm Canny edge detection; by calculating the length, curvature and straightness characteristics of the boundary, the characteristics are cascaded into a long vector which is used as the representation of super pixels and used for graph construction and segmentation;
step 1.3, for each superpixel, calculating the association degree between each superpixel and other superpixels by using a self-attention mechanism, wherein the association degree is regarded as the similarity between the superpixels;
step 1.4, the obtained super-pixel representation and similarity construct a graph, wherein nodes represent super-pixels, and edges represent the similarity between the super-pixels;
step 1.5, defining a cost function, distributing weights to nodes and edges of the graph, measuring similarity and connectivity among superpixels, and distributing attributions of the superpixels, wherein the assignment can be expressed as follows:
E cut ＝∑ (i,j)∈C ω ij
wherein E is cut To sum all sides of graph cut C, ω ij The weight of the edge (i, j) from the pixel i to the pixel j in the graph is given, and C is the set of all edges in the graph;
and 1.6, dividing the graph by using a GraphCut segmentation algorithm, wherein the minimum segmentation is found in the step, and the graph is divided into two parts which respectively represent the foreground and the background.
As a preferable scheme of the spacer image compression method based on the graph segmentation technology power inspection, the invention comprises the following steps: in the step 2, the image dataset is processed based on a segmentation algorithm, and a segmentation result is output, wherein the segmentation result is a segmentation mask with the same size as the input image, each pixel is marked as a different area, the segmentation mask image is divided into 8 x 8 image blocks from left to right and from top to bottom in sequence according to the pixels, if the image size is not a multiple of 8, the image is completed, and the image processing is carried out by taking the 8 x 8 image blocks as a unit; the method specifically comprises the following steps:
step 2.1, starting from the upper left corner, dividing the image into non-overlapping pixel blocks with the size of 8 multiplied by 8 in sequence until the whole image is covered, and if the height or width of the image cannot be divided by 8, filling by using zero;
step 2.2, the sensitivity of human eyes to brightness change is higher than that of color change, image data are processed from the aspects of brightness and color, the image data are converted into YUV color space from RGB color space format according to the following formula, and meanwhile, downsampling is carried out in a mode of Y: U: V=4:2:0; the definition formula is as follows:
step 2.3, performing discrete cosine transform on the 8 multiplied by 8 pixel blocks through two-dimensional DCT, and obtaining a frequency domain matrix through space domain conversion; the DCT decomposes each pixel block into a group of frequency coefficients, wherein the coefficients in the upper left corner represent low frequency information and the coefficients in the lower right corner represent high frequency information, which is a lossless transform;
wherein T is i,j For the DCT matrix calculation formula, M is the matrix width, i, j is the rows and columns of the matrix.
2.4, quantizing DCT coefficients, reducing the fineness of high-frequency information, thereby reducing the data volume, taking the visibility of a target object spacer in a patrol image into consideration, performing lossless operation on 8X 8 pixel blocks related to the target object in graph segmentation, performing high-magnification compression on other pixel blocks, dividing each DCT coefficient by a step length of a corresponding position in a quantization table by using a custom quantization table, rounding, and reducing the precision and the storage space, wherein the larger the quantization step length, the more information loss is caused, but the higher the compression rate is also obtained;
wherein B is i,j For quantized matrices, round () is a rounding function, G i,j For a matrix of luminance components of an image block, Q i,j Is a quantization coefficient matrix;
and 2.5, encoding the quantized coefficients by using an entropy encoding technology to further reduce the size of image data, wherein the coefficients with higher occurrence frequency are encoded into shorter bit sequences and the coefficients with lower occurrence frequency are encoded into longer bit sequences according to the statistical characteristics of the coefficients, so that the data compression is performed efficiently.
As a preferable scheme of the spacer image compression method based on the graph segmentation technology power inspection, the invention comprises the following steps: in the step 3, the codes are compressed into binary code streams, and the binary code streams are stored and transmitted in a format; the encoded data is transmitted through a local area network or a wide area network by using a network transmission protocol, and the encoded JPEG bit stream is transmitted from a storage unit of the inspection equipment to the terminal equipment for further image analysis and processing.
As a preferable scheme of the spacer image compression method based on the graph segmentation technology power inspection, the invention comprises the following steps: after receiving JPEG bit stream data, the terminal equipment decodes according to the inverse process of the coding and restores the JPEG bit stream data into a reconstructed image: entropy decoding- > inverse quantization- > inverse DCT transformation- > color space conversion- > image stitching reorganization, wherein care is taken to discard pixels in order to complement multiples of 8 when dividing the image.
As a preferable scheme of the spacer image compression method based on the graph segmentation technology power inspection, the invention comprises the following steps: in the step 5, the swiniir algorithm is an image restoration algorithm based on Swin transform, and a multi-level attention mechanism is introduced to improve the image restoration algorithm so as to enhance the fusion and reconstruction capability of the characteristic images; the method specifically comprises the following steps:
5.1: collecting a group of model training image pairs, including an image before compression and an image after compression, wherein the data set is divided into two parts, namely an 80% training model and a 20% evaluation performance;
5.2: designing a network architecture of a SwinIR model, establishing a multi-level feature map extraction module of an input image, introducing a partition attention mechanism to perform partition feature fusion after feature extraction, capturing correlations among partitions by using a cross-partition self-attention mechanism, fusing features of different partitions, and introducing high-level semantic information while retaining low-level detail information; in addition, a context sensing mechanism is introduced, and global and local context information is fused in the feature fusion process, so that the network structure can better understand semantic information of the image;
5.3: image reconstruction, namely, after feature output graphs of different levels are fused, inputting the feature output graphs into a reconstruction module, and obtaining a high-resolution recovery image by convolution up-sampling;
5.4: training the SwinIR model using the training dataset, selecting L 1 Optimizing a training model by using a norm loss function, evaluating the performance and effect of the SwinIR model by using an evaluation data set, and selecting a hybrid evaluation model P (peak signal to noise ratio (PSNR) and Structural Similarity Index (SSIM)) to evaluate the reconstruction quality of the model; the relation is:
L 1 ＝||F RHQ -F HQ || 1
P＝αPSNR+(1-α)SSIM
wherein F is RHQ Is a reconstructed high quality image; f (F) HQ Alpha E [0,1 ] for a high quality map corresponding to low quality]Constant of the same.
In a second aspect, a spacer image compression system based on graph segmentation technology power inspection is provided, which is configured to implement the spacer image compression method based on graph segmentation technology power inspection as described above, and is characterized in that: the method comprises the following modules:
the data acquisition module is used for acquiring a spacer image of the inspection equipment and acquiring an inspection image with high resolution through the inspection equipment with a camera;
the image segmentation technology module is used for carrying out segmentation processing on the background and the target area in the spacer image of the inspection equipment;
the image data compression module is used for respectively setting different compression rates of a target area and a background area in an image, coding and processing the target area and the background area into a binary code stream;
the image data transmission module is used for transmitting the coded image data through a network protocol and transmitting the coded image data to the terminal equipment from the storage unit of the inspection equipment;
the image data reconstruction module is used for decoding the coded data received by the terminal equipment so as to restore the compressed image;
and the super-resolution enhancement module is used for performing super-resolution enhancement processing on the image data based on the SwinIR algorithm.
In a third aspect, an electronic device is provided, comprising a memory and a processor, the memory having stored therein computer program instructions which, when read and executed by the processor, perform the steps of the method described above.
In a fourth aspect, a computer-readable storage medium is provided, wherein: the computer readable storage medium has stored thereon computer program instructions which, when read and executed by a processor, perform the steps of the method described above.
The invention has the beneficial effects that: according to the invention, through graph segmentation and different compression rate selection, the target pixel of the image is extracted, redundant information is reduced, the image can be compressed more accurately, and a higher compression rate is realized, so that the data volume of storage and transmission is reduced; the key details and important information in the image can be kept as much as possible based on the data compression of the image segmentation and the reconstruction of the super-segmentation algorithm, so that the compressed image has a more visual effect; the graph segmentation technique can reduce the complexity of the image, so that the subsequent compression and processing operations are more efficient, and the computing resource and time cost are saved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
fig. 1 is a process flow diagram of a spacer image compression method based on a graph segmentation technique for power inspection.
Fig. 2 is a schematic diagram of a graph cut algorithm of a spacer image compression method based on graph cut technology power inspection.
Fig. 3 is a coding and decoding flow chart of the JPEG compression technique of the spacer image compression method based on the power inspection of the graph segmentation technique.
Fig. 4 is a flowchart of super-division enhanced image processing of a spacer image compression method based on the graph-division technology power inspection.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
Example 1
At present, in the inspection of a power line, a large amount of image data can be acquired by inspection equipment, and the conventional image compression method cannot meet the dual requirements on image quality and data transmission efficiency. Therefore, it is necessary to compress and process the images in different proportions according to the data with different importance levels in the images, so as to improve the image compression efficiency, reduce the data transmission amount, and ensure the visual quality of the compressed images.
Some image compression methods exist at present, and have problems of information loss and low compression ratio when processing high-resolution images. The inspection device spacer inspection image is characterized in that an obvious segmentation boundary exists between the background and the target.
Referring to fig. 1 to 4, in a first embodiment of the present invention, a spacer image compression method based on a graph segmentation technique for power inspection is provided, which can effectively distinguish between foreground and background areas of an image by adopting the compression method based on the graph segmentation technique, and combine the importance of different areas in the image to perform lossy or lossless compression, so that efficient compression of the image is realized on the premise of meeting the image quality requirement, and the transmission amount and storage space of image data are reduced.
Specifically, the method comprises the following steps:
step 1, acquiring a spacer image acquired by inspection equipment, and carrying out segmentation processing on a background and a target area in the image;
the method includes the steps that a spacer image of a patrol equipment is obtained, a patrol image with high resolution is obtained through the patrol equipment with a camera, a spacer in the patrol image is used as a target based on a graph segmentation technology, segmentation outlines of the target and a background are found, and preparation is made for compression of different multiplying powers for subsequent target pixel blocks and non-target pixel blocks.
Step 2, respectively setting different compression rates for a target area and a background area in an image, and encoding based on a JPEG compression technology;
it should be noted that, the segmentation processing is performed on the wire and the background by using the pattern cutting technology in combination with the scene.
Step 3, storing JPEG encoded data and transmitting the JPEG encoded data to a terminal;
step 4, the terminal decodes the coded data based on the JPEG compression technology to restore the compressed image;
step 5, performing super-division enhancement on the image by a SwinIR algorithm based on a multi-level attention mechanism;
and 6, obtaining the enhanced high-visualization compressed image.
Furthermore, the step 1 further comprises preprocessing the acquired spacer images, including adjusting the size, marking targets, and amplifying image data in a rotating and overturning mode, wherein the data set is divided into a training set and a verification set.
It should be noted that increasing the diversity of the data set by rotation, flipping, etc. helps to improve the generalization ability of the model, and the data set is divided into a training set and a verification set so as to evaluate the performance of the model when training.
Further, referring to fig. 2, the segmentation processing is to segment the target region and the background region based on feature cascade and attention mechanism; the method specifically comprises the following steps:
step 1, dividing an input image into a plurality of connected uniform superpixels by using a SLIC (Simple Linear Iterative Clustering) superpixel generation algorithm, reducing the complexity of subsequent image processing, and retaining main structural information of the image;
step 2, performing feature cascade analysis, and calculating the color, texture, shape and other features of each super pixel; calculating the average value of all pixel colors in the super pixel to obtain a vector representing the color; calculating texture features of pixels within the superpixel using the texture descriptors; obtaining the boundary information of the super pixel by using a boundary contour extraction algorithm Canny edge detection; by calculating the length, curvature and straightness characteristics of the boundary, the characteristics are cascaded into a long vector which is used as the representation of super pixels and used for graph construction and segmentation;
it should be noted that, statistical information of the texture, such as texture energy, contrast, uniformity, etc., is counted.
Step 3, for each super pixel, calculating the association degree between each super pixel and other super pixels by using a self-attention mechanism, wherein the association degree is regarded as the similarity between the super pixels;
preferably, the self-attention mechanism allows each superpixel to adjust its associated weight with other superpixels based on its own characteristics, thereby better capturing local relationships in the image.
Step 4, the obtained super-pixel representation and similarity construct a graph, wherein nodes represent super-pixels, and edges represent the similarity between the super-pixels;
preferably, such a graph structure helps to represent pixel relationships and similarities in images.
Step 5, defining a cost function, distributing weights to nodes and edges of the graph, measuring similarity and connectivity between superpixels, and distributing attributions of the superpixels, wherein the assignment can be expressed as follows:
E cut ＝∑ (i,j)∈C ω ij
wherein E is cut To sum all sides of graph cut C, ω ij The weight of the edge (i, j) from the pixel i to the pixel j in the graph is given, and C is the set of all edges in the graph;
and 6, dividing the graph by using a GraphCut segmentation algorithm, wherein the minimum segmentation is found in the step, and the graph is divided into two parts which respectively represent the foreground and the background.
Preferably, the algorithm aims to find a minimal cut, dividing the graph into two parts, representing the foreground and the background, respectively. By minimizing the defined cost function, effective image segmentation can be realized, so that clear foreground and background areas are obtained, and the description is omitted in the prior art.
Further, referring to fig. 3, the image data set is processed based on the segmentation algorithm, and the segmentation result is output, wherein the segmentation result is a segmentation mask having the same size as the input image, each pixel is marked as a different region, the segmentation mask image is divided into 8×8 image blocks sequentially from left to right and from top to bottom according to the pixels, if the image size is not a multiple of 8, the image is completed, and the image processing is performed in units of 8×8 image blocks.
Different compression rates are respectively set for a target area and a background area in an image, and coding is carried out based on a JPEG compression technology, specifically comprising the following steps:
step 1, starting from the upper left corner, dividing an image into non-overlapping pixel blocks with the size of 8 multiplied by 8 in sequence until the whole image is covered, and if the height or width of the image cannot be divided by 8, filling by using zero;
this step is to ensure that the entire image is covered.
Step 2, the sensitivity of human eyes to brightness change is higher than that of color change, image data are processed from the aspects of brightness and color, the image data are converted into YUV color space from RGB color space format according to the following formula, and meanwhile, downsampling is carried out in a mode of Y: U: V=4:2:0; the definition formula is as follows:
it should be noted that this is because the human eye is more sensitive to the change of brightness, and the conversion method is more consistent with the visual characteristics.
Step 3, performing Discrete Cosine Transform (DCT) on the 8 multiplied by 8 pixel blocks through two-dimensional DCT, and obtaining a frequency domain matrix through space domain conversion; the DCT decomposes each pixel block into a group of frequency coefficients, wherein the coefficients in the upper left corner represent low frequency information and the coefficients in the lower right corner represent high frequency information, which is a lossless transform;
wherein T is i,j For the DCT matrix calculation formula, M is the matrix width, i, j is the rows and columns of the matrix;
step 4, quantizing DCT coefficients, reducing the fineness of high-frequency information, thereby reducing the data volume, carrying out lossless operation on 8X 8 pixel blocks related to a target object in image segmentation in consideration of the visibility of the target object in an inspection image, carrying out high-magnification compression on other pixel blocks, dividing each DCT coefficient by the step length of the corresponding position in the quantization table and rounding off, and reducing the precision and the storage space, wherein the larger the quantization step length is, the more information loss is caused, but the higher the compression rate is also obtained;
wherein B is i,j For quantized matrices, round () is a rounding function, G i,j For a matrix of luminance components of an image block, Q i,j Is a quantization coefficient matrix;
and 5, encoding the quantized coefficients by using an entropy encoding technology to further reduce the size of the image data, wherein the coefficients with higher occurrence frequency are encoded into a shorter bit sequence and the coefficients with lower occurrence frequency are encoded into a longer bit sequence according to the statistical characteristics of the coefficients, so that the data compression is performed efficiently.
Further, the JPEG encoded data is stored and transmitted to a terminal step, encoded to be compressed into a binary code stream, and stored and transmitted in a binary code stream format; the encoded data is transmitted through a local area network or a wide area network by using a network transmission protocol, and the encoded JPEG bit stream is transmitted from a storage unit of the inspection equipment to the terminal equipment for further image analysis and processing.
It should be noted that the JPEG encoded data is transmitted over a local area network or a wide area network using a network transmission protocol such as TCP/IP.
Preferably, the terminal device may be a computer, a mobile phone, a tablet computer or other devices supporting image analysis and processing.
Further, the terminal decodes the encoded data based on the JPEG compression technology, so that in the step of restoring the compressed image, the terminal equipment decodes the encoded data according to the inverse process of encoding after receiving the JPEG bit stream data, and restores the encoded data into a reconstructed image; the order is entropy decoding, inverse quantization, inverse DCT transformation, color space conversion, and image stitching reorganization, where care is taken to discard pixels that are multiples of 8 to complement when the image is segmented.
Further, referring to fig. 4, in the step of performing super-resolution enhancement on the image by using a swiniir algorithm based on a multi-level attention mechanism, the swiniir algorithm is an image restoration algorithm based on a Swin transducer, and the multi-level attention mechanism is introduced and improved to enhance the fusion and reconstruction capability of the feature image; the method specifically comprises the following steps:
step 1, collecting a group of model training image pairs, including an image before compression and an image after compression, wherein the data set is divided into two major parts, namely an 80% training model and 20% evaluation performance;
step 2, designing a network architecture of a SwinIR model, establishing a multi-level feature map extraction module of an input image, introducing a partition attention mechanism to perform partition feature fusion after feature extraction, capturing the correlation between partitions by using a cross-partition self-attention mechanism, fusing the features of different partitions, and introducing high-level semantic information while retaining low-level detail information; in addition, a context sensing mechanism is introduced, and global and local context information is fused in the feature fusion process, so that the network structure can better understand semantic information of the image;
step 3, reconstructing the image, namely fusing the characteristic output images of different levels, inputting the fused characteristic output images into a reconstruction module, and obtaining a high-resolution recovery image by using convolution up-sampling;
step 4 training the SwinIR model by using the training data set, and selecting L 1 Optimizing a training model by using a norm loss function, evaluating the performance and effect of the SwinIR model by using an evaluation data set, and selecting a hybrid evaluation model P (peak signal to noise ratio (PSNR) and Structural Similarity Index (SSIM)) to evaluate the reconstruction quality of the model; the relation is:
L 1 ＝||F RHQ -F HQ || 1
P＝αPSNR+(1-α)SSIM
wherein F is RHQ Is a reconstructed high quality image; f (F) HQ Alpha E [0,1 ] for a high quality map corresponding to low quality]Constant of the same.
Example 2
A second embodiment of the present invention, which is different from the first embodiment, is: the spacer image compression system based on the graph segmentation technology power inspection is used for realizing a spacer image compression method based on the graph segmentation technology power inspection. The method specifically comprises the following modules:
the data acquisition module is used for acquiring a spacer image of the inspection equipment and acquiring an inspection image with high resolution through the inspection equipment with a camera;
the image segmentation technology module is used for carrying out segmentation processing on the background and the target area in the spacer image of the inspection equipment;
the image data compression module is used for respectively setting different compression rates of a target area and a background area in an image, coding and processing the target area and the background area into a binary code stream;
the image data transmission module is used for transmitting the coded image data through a network protocol and transmitting the coded image data to the terminal equipment from the storage unit of the inspection equipment;
the image data reconstruction module is used for decoding the coded data received by the terminal equipment so as to restore the compressed image;
and the super-resolution enhancement module is used for performing super-resolution enhancement processing on the image data based on the SwinIR algorithm.
Further, the embodiment of the invention also provides a computer readable storage medium, on which computer program instructions are stored, and when the computer program instructions are read and run by a processor, the steps of the spacer image compression method based on the graph segmentation technology power inspection provided by the embodiment of the invention are executed.
Further, the embodiment of the invention also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are read and run by the processor, the steps of the spacer image compression method based on the graph segmentation technology power inspection provided by the embodiment of the invention are executed.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (10)
1. A spacer image compression method based on graph segmentation technology power inspection is characterized in that: the method comprises the following steps:
step 1, acquiring a spacer image acquired by inspection equipment, and carrying out segmentation processing on a background and a target area in the image;
step 2, respectively setting different compression rates for a target area and a background area in an image, and encoding based on a JPEG compression technology;
step 3, storing the coded data and transmitting the coded data to a terminal;
step 4, the terminal decodes the coded data based on the JPEG compression technology to restore the compressed image;
step 5, performing super-division enhancement on the image by a SwinIR algorithm based on a multi-level attention mechanism;
and 6, obtaining the enhanced high-visualization compressed image.
2. The spacer image compression method based on graph segmentation technology power inspection of claim 1, wherein the method is characterized by comprising the following steps: the step 1 also comprises preprocessing the acquired spacer images, including adjusting the size, marking targets, and amplifying image data in a rotating and overturning mode, wherein the data set is divided into a training set and a verification set.
3. The spacer image compression method based on graph segmentation technology power inspection according to claim 2, wherein the method is characterized in that: in the step 1, a segmentation technology based on feature cascade and an attention mechanism carries out segmentation processing on a target area and a background area; the method specifically comprises the following steps:
step 1.1, dividing an input image into a plurality of connected uniform superpixels by using an SLIC superpixel generation algorithm, reducing the complexity of subsequent image processing, and retaining main structural information of the image;
step 1.2, performing feature cascade analysis, and calculating the color, texture and shape features of each super pixel; calculating the average value of all pixel colors in the super pixel to obtain a vector representing the color; calculating texture features of pixels within the superpixel using the texture descriptors; obtaining the boundary information of the super pixel by using a boundary contour extraction algorithm Canny edge detection; by calculating the length, curvature and straightness characteristics of the boundary, the characteristics are cascaded into a long vector which is used as the representation of super pixels and used for graph construction and segmentation;
step 1.3, for each superpixel, calculating the association degree between each superpixel and other superpixels by using a self-attention mechanism, wherein the association degree is regarded as the similarity between the superpixels;
step 1.4, the obtained super-pixel representation and similarity construct a graph, wherein nodes represent super-pixels, and edges represent the similarity between the super-pixels;
step 1.5, defining a cost function, distributing weights to nodes and edges of the graph, measuring similarity and connectivity among superpixels, and distributing attributions of the superpixels, wherein the assignment can be expressed as follows:
E cut ＝∑ (i,j)∈C ω ij
wherein E is cut To sum all sides of graph cut C, ω ij The weight of the edge (i, j) from the pixel i to the pixel j in the graph is given, and C is the set of all edges in the graph;
and 1.6, dividing the graph by using a GraphCut segmentation algorithm, wherein the minimum segmentation is found in the step, and the graph is divided into two parts which respectively represent the foreground and the background.
4. The spacer image compression method based on graph segmentation technology power inspection according to claim 3, wherein the method comprises the following steps: in the step 2, the image dataset is processed based on the segmentation algorithm, and a segmentation result is output, wherein the segmentation result is a segmentation mask with the same size as the input image, each pixel is marked as a different area, the segmentation mask image is divided into 8 x 8 image blocks from left to right and from top to bottom in sequence according to the pixels, if the image size is not a multiple of 8, the image is completed, and the image processing is performed by taking the 8 x 8 image blocks as a unit; the method specifically comprises the following steps:
step 2.1, starting from the upper left corner, dividing the image into non-overlapping pixel blocks with the size of 8 multiplied by 8 in sequence until the whole image is covered, and if the height or width of the image cannot be divided by 8, filling by using zero;
step 2.2, the sensitivity of human eyes to brightness change is higher than that of color change, image data are processed from the aspects of brightness and color, the image data are converted into YUV color space from RGB color space format according to the following formula, and meanwhile, downsampling is carried out in a mode of Y: U: V=4:2:0; the definition formula is as follows:
step 2.3, performing discrete cosine transform on the 8 multiplied by 8 pixel blocks through two-dimensional DCT, and obtaining a frequency domain matrix through space domain conversion; the DCT decomposes each pixel block into a group of frequency coefficients, wherein the coefficients in the upper left corner represent low frequency information and the coefficients in the lower right corner represent high frequency information, which is a lossless transform;
wherein T is i,j For the DCT matrix calculation formula, M is the matrix width, i, j is the rows and columns of the matrix;
2.4, quantizing DCT coefficients, reducing the fineness of high-frequency information, thereby reducing the data volume, taking the visibility of a target object spacer in a patrol image into consideration, performing lossless operation on 8X 8 pixel blocks related to the target object in graph segmentation, performing high-magnification compression on other pixel blocks, dividing each DCT coefficient by a step length of a corresponding position in a quantization table by using a custom quantization table, rounding, and reducing the precision and the storage space, wherein the larger the quantization step length, the more information loss is caused, but the higher the compression rate is also obtained;
wherein B is i,j For quantized matrices, round () is a rounding function, G i,j For a matrix of luminance components of an image block, Q i,j Is a quantization coefficient matrix;
and 2.5, encoding the quantized coefficients by using an entropy encoding technology to further reduce the size of the image data, and performing entropy encoding by using Huffman encoding, wherein the coefficients with higher occurrence frequency are encoded into shorter bit sequences and the coefficients with lower occurrence frequency are encoded into longer bit sequences according to the statistical characteristics of the coefficients, so that the data compression is performed efficiently.
5. The spacer image compression method based on graph segmentation technology power inspection of claim 4, wherein the method comprises the following steps: in the step 3, the codes are compressed into binary code streams, and the binary code streams are stored and transmitted in a format; the encoded data is transmitted through a local area network or a wide area network by using a network transmission protocol, and the encoded JPEG bit stream is transmitted from a storage unit of the inspection equipment to the terminal equipment for further image analysis and processing.
6. The spacer image compression method based on graph segmentation technology power inspection of claim 4, wherein the method comprises the following steps: after receiving JPEG bit stream data, the terminal equipment decodes according to the inverse process of the coding and restores the reconstructed image; the order is entropy decoding, inverse quantization, inverse DCT transformation, color space conversion, and image stitching reorganization, where care is taken to discard pixels that are multiples of 8 to complement when the image is segmented.
7. The spacer image compression method based on graph segmentation technology power inspection of claim 1, wherein the method is characterized by comprising the following steps: in the step 5, the swiniir algorithm is an image restoration algorithm based on Swin transform, and a multi-level attention mechanism is introduced to improve the image restoration algorithm so as to enhance the fusion and reconstruction capability of the characteristic images; the method specifically comprises the following steps:
step 5.1, collecting a group of model training image pairs, including an image before compression and an image after compression, wherein the data set is divided into two major parts of 80% training models and 20% evaluation performances;
step 5.2, designing a network architecture of a SwinIR model, establishing a multi-level feature map extraction module of an input image, introducing a partition attention mechanism to perform partition feature fusion after feature extraction, capturing the correlation between partitions by using a cross-partition self-attention mechanism, fusing the features of different partitions, and introducing high-level semantic information while retaining low-level detail information; in addition, a context sensing mechanism is introduced, and global and local context information is fused in the feature fusion process, so that the network structure can better understand semantic information of the image;
step 5.3, reconstructing the image, namely fusing the characteristic output images of different levels, inputting the fused characteristic output images into a reconstruction module, and obtaining a high-resolution recovery image by using convolution up-sampling;
step 5.4, training the SwinIR model by using the training data set, and selecting L 1 Optimizing a training model by using a norm loss function, evaluating the performance and effect of the SwinIR model by using an evaluation data set, and selecting a hybrid evaluation model P (peak signal to noise ratio (PSNR) and Structural Similarity Index (SSIM)) to evaluate the reconstruction quality of the model; the relation is:
L 1 ＝||F RHQ -F HQ || 1
P＝αPSNR+(1-α)SSIM
wherein F is RHQ Is a reconstructed high quality image; f (F) HQ Alpha E [0,1 ] for a high quality map corresponding to low quality]Constant of the same.
8. A spacer image compression system based on graph segmentation technology power inspection, for implementing the spacer image compression method based on graph segmentation technology power inspection according to any one of claims 1 to 7, characterized in that: the method comprises the following modules:
the data acquisition module is used for acquiring a spacer image of the inspection equipment and acquiring an inspection image with high resolution through the inspection equipment with a camera;
the image segmentation technology module is used for carrying out segmentation processing on the background and the target area in the spacer image of the inspection equipment;
the image data compression module is used for respectively setting different compression rates of a target area and a background area in an image, coding and processing the target area and the background area into a binary code stream;
the image data transmission module is used for transmitting the coded image data through a network protocol and transmitting the coded image data to the terminal equipment from the storage unit of the inspection equipment;
the image data reconstruction module is used for decoding the coded data received by the terminal equipment so as to restore the compressed image;
and the super-resolution enhancement module is used for performing super-resolution enhancement processing on the image data based on the SwinIR algorithm.
9. An electronic device comprising a memory and a processor, the memory having stored therein computer program instructions, characterized in that the computer program instructions, when read and executed by the processor, perform the steps of the method of any of claims 1-7.
10. A computer readable storage medium, characterized in that it has stored thereon computer program instructions which, when read and run by a processor, perform the steps of the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311631234.1A CN117560511A (en) | 2023-11-27 | 2023-11-27 | Spacer image compression method and system based on graph segmentation technology and electric power inspection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311631234.1A CN117560511A (en) | 2023-11-27 | 2023-11-27 | Spacer image compression method and system based on graph segmentation technology and electric power inspection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117560511A true CN117560511A (en) | 2024-02-13 |
Family
ID=89823088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311631234.1A Pending CN117560511A (en) | 2023-11-27 | 2023-11-27 | Spacer image compression method and system based on graph segmentation technology and electric power inspection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117560511A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117808721A (en) * | 2024-02-28 | 2024-04-02 | 深圳市瓴鹰智能科技有限公司 | Low-illumination image enhancement method, device, equipment and medium based on deep learning |
-
2023
- 2023-11-27 CN CN202311631234.1A patent/CN117560511A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117808721A (en) * | 2024-02-28 | 2024-04-02 | 深圳市瓴鹰智能科技有限公司 | Low-illumination image enhancement method, device, equipment and medium based on deep learning |
| CN117808721B (en) * | 2024-02-28 | 2024-05-03 | 深圳市瓴鹰智能科技有限公司 | Low-illumination image enhancement method, device, equipment and medium based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6476805B1 (en) | Techniques for spatial displacement estimation and multi-resolution operations on light fields | |
| CN108028941B (en) | Method and apparatus for encoding and decoding digital images by superpixel | |
| CN108921910B (en) | JPEG coding compressed image restoration method based on scalable convolutional neural network | |
| CN110830803B (en) | Image compression method combining block matching and string matching | |
| CN109842803A (en) | A kind of method and device of compression of images | |
| Yadav et al. | A review on image compression techniques | |
| CN117560511A (en) | Spacer image compression method and system based on graph segmentation technology and electric power inspection | |
| CN105100814A (en) | Methods and devices for image encoding and decoding | |
| CN111800630A (en) | Method and system for reconstructing video super-resolution and electronic equipment | |
| Hu et al. | An adaptive two-layer light field compression scheme using GNN-based reconstruction | |
| CN104869426A (en) | JPEG coding method lowering image diamond effect under low compression code rate | |
| Kekre et al. | Image Reconstruction using Fast Inverse Half tone and Huffman Coding Technique | |
| CN113194312B (en) | Planetary science exploration image adaptive quantization coding system combined with visual saliency | |
| CN115209147A (en) | Camera video transmission bandwidth optimization method, device, equipment and storage medium | |
| CN113422965A (en) | Image compression method and device based on generation countermeasure network | |
| CN107155111B (en) | Video compression method and device | |
| JP2003304537A (en) | Method of processing digital image for low-bit rate application | |
| CN111031312B (en) | Image compression method for realizing attention mechanism based on network | |
| CN116684607B (en) | Image compression and decompression method and device, electronic equipment and storage medium | |
| WO2024119420A1 (en) | Encoding method, decoding method, code stream, encoder, decoder, and storage medium | |
| Jain et al. | Enhanced SVD Based Image Compression Technique | |
| Baluja et al. | Neural image decompression: Learning to render better image previews | |
| CN105007490A (en) | OmapL138-chip-based Jpeg compression algorithm | |
| Singh et al. | COMPARATIVE STUDIES OF VARIOUS TECHNIQUES FOR IMAGE COMPRESSION ALGORITHM | |
| CN113989398A (en) | CT data lossless compression and decompression method and system, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |