CN117115465A - Multi-channel high-resolution edge fusion method - Google Patents
Multi-channel high-resolution edge fusion method Download PDFInfo
- Publication number
- CN117115465A CN117115465A CN202311113762.8A CN202311113762A CN117115465A CN 117115465 A CN117115465 A CN 117115465A CN 202311113762 A CN202311113762 A CN 202311113762A CN 117115465 A CN117115465 A CN 117115465A
- Authority
- CN
- China
- Prior art keywords
- edge
- image
- texture feature
- resolution
- feature map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title description 5
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000002156 mixing Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 25
- 230000004927 fusion Effects 0.000 claims abstract description 20
- 239000013598 vector Substances 0.000 claims description 31
- 238000009826 distribution Methods 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 21
- 238000003708 edge detection Methods 0.000 claims description 14
- 238000005457 optimization Methods 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 6
- 238000003860 storage Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 238000012935 Averaging Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 6
- 230000014509 gene expression Effects 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/86—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using syntactic or structural representations of the image or video pattern, e.g. symbolic string recognition; using graph matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
A multi-channel high resolution edge blending method is disclosed. Firstly, acquiring a low-resolution image, then, carrying out edge information enhancement processing on the low-resolution image to obtain an edge information enhanced texture feature map, and then, generating a multi-channel high-resolution edge fusion image based on the edge information enhanced texture feature map. In this way, edge information can be extracted and fused efficiently.
Description
Technical Field
The present disclosure relates to the field of edge blending, and more particularly, to a multi-channel high resolution edge blending method.
Background
Image resolution refers to the number of pixels contained in an image, which determines the sharpness and level of detail of the image. High resolution images have more pixels and thus can display more detail, but also require more memory and computational resources. Low resolution images, in contrast, take up less space and resources, but lose some detail. In some application scenarios, such as medical imaging, remote sensing imaging, video surveillance, etc., a low resolution image needs to be enlarged or reconstructed to obtain a high resolution image. This is the task of image super-resolution reconstruction.
The goal of image super-resolution reconstruction is to recover a high resolution image from one or more low resolution images with higher definition and more rich detail. This is a very challenging problem because there are factors such as information loss, noise, blurring, etc. in the low resolution image, resulting in uncertainty and multiple solutions in the reconstruction of the high resolution image. Many conventional methods, such as interpolation, do not have ideal image reconstruction effects and are prone to aliasing and blurring effects.
Thus, an optimized solution is desired.
Disclosure of Invention
In view of this, the present disclosure proposes a multi-channel high-resolution edge fusion method that can efficiently extract and fuse edge information.
According to an aspect of the present disclosure, there is provided a multi-channel high resolution edge blending method, including:
acquiring a low resolution image;
performing edge information enhancement processing on the low-resolution image to obtain an edge information enhancement texture feature map; and
and generating a multi-channel high-resolution edge fusion image based on the edge information enhanced texture feature map.
According to the embodiment of the disclosure, a low-resolution image is firstly acquired, then edge information enhancement processing is carried out on the low-resolution image to obtain an edge information enhancement texture feature map, and then a multi-channel high-resolution edge fusion image is generated based on the edge information enhancement texture feature map. In this way, edge information can be extracted and fused efficiently.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 shows a flow chart of a multi-channel high resolution edge blending method according to an embodiment of the present disclosure.
Fig. 2 shows an architecture schematic of a multi-channel high resolution edge blending method according to an embodiment of the present disclosure.
Fig. 3 shows a flowchart of sub-step S120 of a multi-channel high resolution edge blending method according to an embodiment of the present disclosure.
Fig. 4 shows a flowchart of sub-step S122 of the multi-channel high resolution edge blending method according to an embodiment of the present disclosure.
Fig. 5 shows a flowchart of sub-step S130 of a multi-channel high resolution edge blending method according to an embodiment of the present disclosure.
Fig. 6 illustrates a block diagram of a multi-channel high resolution edge blending system according to an embodiment of the present disclosure.
Fig. 7 illustrates an application scenario diagram of a multi-channel high-resolution edge blending method according to an embodiment of the present disclosure.
Detailed Description
The following description of the embodiments of the present disclosure will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the disclosure. All other embodiments, which can be made by one of ordinary skill in the art without undue burden based on the embodiments of the present disclosure, are also within the scope of the present disclosure.
As used in this disclosure and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
It should be understood that edge information is one of the most important features in an image, and reflects information such as the contour, shape, texture, etc. of an object. In low resolution images, edge information tends to be incomplete or unclear due to information loss, noise, blurring, and the like. If the low resolution image is directly enlarged, the edge information becomes more blurred or distorted. If the conventional deep learning method is used, the edge information may be ignored or suppressed due to the limitation of the network structure and the loss function. Therefore, how to effectively extract and fuse the edge information in the image resolution reconstruction is a problem to be solved.
In view of the above technical problems, the technical idea of the present disclosure is to extract image texture features in a low-resolution image by using a multi-scale convolution structure, and perform edge detection and weighting on the image texture features, and in this way, enhance edge information in feature distribution. Then, the texture feature map is converted into a high resolution image using an AIGC-based high resolution image generator while maintaining or enhancing edge information.
Based on this, fig. 1 shows a flowchart of a multi-channel high-resolution edge blending method according to an embodiment of the present disclosure. Fig. 2 shows an architecture schematic of a multi-channel high resolution edge blending method according to an embodiment of the present disclosure. As shown in fig. 1 and 2, a multi-channel high-resolution edge blending method according to an embodiment of the present disclosure includes the steps of: s110, acquiring a low-resolution image; s120, carrying out edge information enhancement processing on the low-resolution image to obtain an edge information enhanced texture feature map; and S130, generating a multi-channel high-resolution edge fusion image based on the edge information enhanced texture feature map. It should be appreciated that in step S110, the low resolution image is acquired for providing input for subsequent processing and edge blending, and may be a downsampled version of the original image, typically with a lower pixel density and detail information. In step S120, edge information enhancement processing is performed on the low resolution image, in order to highlight edge features in the image, and the edge information enhancement may be implemented by various image processing technologies, for example, an edge detection algorithm, and the intensity and sharpness of the edge may be improved, so that edge fusion may be better performed in a subsequent step. In step S130, a multi-channel high-resolution edge fusion image is generated using the texture feature map subjected to the edge information enhancement processing, and the multi-channel edge fusion may be performed by fusing the texture feature map with the original low-resolution image to enhance details and edge information of the image, which may involve techniques such as image interpolation, filtering, weighted fusion, etc., to generate an image with higher resolution and sharper edges. The steps of the multi-channel high-resolution edge fusion method are to obtain a low-resolution image, enhance edge information and texture characteristics, and fuse multi-channel information to finally generate an image with high resolution and clear edges, wherein each step has a specific function so as to achieve a final edge fusion effect.
Specifically, in the technical scheme of the present disclosure, first, a low-resolution image is acquired, and the low-resolution image is passed through an image texture feature extractor having a multi-scale convolution structure to obtain an image texture feature map. Here, since the texture features of the image have different representation capabilities on different scales, in the technical solution of the present disclosure, it is desirable to capture the texture feature distribution in different spatial neighbors using a multi-scale convolution structure.
Then, edge features in the image texture feature map are extracted to obtain edge weight vectors. In a specific example of the present disclosure, the encoding process for extracting edge features in the image texture feature map to obtain edge weight vectors includes: firstly, carrying out edge detection on each feature matrix in the image texture feature graph to obtain a sequence of edge feature matrices; and carrying out global mean pooling on each feature matrix in the sequence of the edge feature matrix to obtain an edge weight vector.
Accordingly, as shown in fig. 3, performing edge information enhancement processing on the low-resolution image to obtain an edge information enhanced texture feature map, including: s121, carrying out multi-scale feature extraction on the low-resolution image to obtain an image texture feature map; s122, extracting edge features in the image texture feature map to obtain edge weight vectors; and S123, weighting the image texture feature map by taking the edge weight vector as a weight vector to obtain the edge information enhanced texture feature map. It should be appreciated that in step S121, multi-scale feature extraction is performed on the low resolution image in order to extract texture features of different scales from the image, and multi-scale feature extraction may be implemented by using filters or transforms of different scales (e.g., wavelet transforms), so that details and texture information in the image may be captured, providing a richer feature representation for subsequent steps. In step S122, edge features are extracted from the image texture feature map to obtain edge weight vectors, and the edge feature extraction may use an edge detection algorithm, such as Canny edge detection or Sobel operator, and the extracted edge features may be used to weight the texture features in a subsequent step. In step S123, the image texture feature map is weighted using the previously extracted edge weight vector as a weight vector to obtain an edge information enhanced texture feature map, and by applying the edge weight vector to each pixel of the texture feature map, the texture details of the edge region can be highlighted while suppressing the texture features of the non-edge region, thereby enhancing the edge information. In other words, through the three steps S121, S122, and S123, multi-scale feature extraction may be performed on the low-resolution image, edge features are extracted and edge weight vectors are generated, and then the weighting process is performed on the image texture feature map using the weight vectors, thereby obtaining an edge information enhanced texture feature map. These steps help to extract and enhance the edge information of the image, providing more discernable and sharp features for the subsequent edge blending process.
More specifically, in step S121, multi-scale feature extraction is performed on the low-resolution image to obtain an image texture feature map, including: and passing the low-resolution image through an image texture feature extractor with a multi-scale convolution structure to obtain the image texture feature map. It should be noted that the multi-scale convolution structure is a design method of a convolutional neural network, which is used to extract feature representations of multiple scales from an input image, and is implemented by introducing convolution layers with different receptive fields (receptive fields) into the network. The image texture feature extractor with the multi-scale convolution structure can carry out convolution operation on the input image on different scales so as to capture texture features of different scales in the image. Each scale convolution layer may have a different convolution kernel size or stride to accommodate different scale features. Smaller convolution kernels may capture detail texture features, while larger convolution kernels may capture broader structural features. The main purpose of the multi-scale convolution structure is to extract multi-scale feature information from an image, thereby more fully representing the texture features of the image. By performing convolution operations on different scales, the network can focus on local details and global structures at the same time, thereby improving the perceptibility of image textures. This is very useful for edge information enhancement processing, as edges typically have features of different dimensions. The multi-scale convolution structure allows the image texture feature extractor to convolve the low resolution image at different scales to obtain a multi-scale texture feature representation. This helps to extract detail and structural information from the image, providing a richer and comprehensive representation of features for subsequent edge information enhancement and edge fusion steps.
More specifically, in step S122, as shown in fig. 4, extracting edge features in the image texture feature map to obtain an edge weight vector includes: s1221, performing edge detection on each feature matrix in the image texture feature graph to obtain a sequence of edge feature matrices; and S1222, carrying out global averaging pooling on each feature matrix in the sequence of the edge feature matrix to obtain the edge weight vector. It is worth mentioning that edge detection is an image processing technique for detecting edge regions in an image, which determines the edge position in the image by analyzing intensity variations or gradient information in the image, the edges typically representing the boundaries between different regions in the image or the contours of an object. The purpose of extracting edge features in an image texture feature map is to identify and capture edge information in the image so that subsequent steps can be weighted according to these edge features. In step S1221, an edge detection operation is performed on each feature matrix in the image texture feature map to obtain a sequence of edge feature matrices, and the edge detection algorithm may use various methods, for example, canny edge detection, sobel operator, laplacian operator, etc., and these algorithms may determine the position of an edge by calculating the change or gradient of the pixel intensity in the image, so as to generate an edge feature matrix sequence. In step S1222, global averaging is performed on each feature matrix in the sequence of edge feature matrices to obtain an edge weight vector, where global averaging is a pooling operation that averages the values of each channel of the feature matrices to obtain a single value as a summary representation of the channel, and by performing global averaging on each feature matrix in the sequence of edge feature matrices, information of each feature matrix may be summarized into an edge weight value to form an edge weight vector. In other words, through the two steps of S1221 and S1222, the edge feature matrix sequence can be extracted from the image texture feature map and converted into an edge weight vector through the global averaging pooling operation. These edge weight vectors will be used as weights for weighting the image texture feature map in subsequent steps to enhance the edge information and generate an edge information enhanced texture feature map. The purpose of edge detection is to extract and emphasize edge structures in the image, providing more accurate and meaningful weights for subsequent edge information enhancement and fusion.
Then, weighting the image texture feature map by taking the edge weight vector as a weight vector to obtain an edge information enhanced texture feature map; and the edge information enhanced texture feature image passes through an AIGC-based high-resolution image generator to obtain a multi-channel high-resolution edge fusion image.
Accordingly, as shown in fig. 5, generating a multi-channel high-resolution edge fusion image based on the edge information enhanced texture feature map includes: s131, performing feature distribution optimization on the edge information enhancement texture feature map to obtain an optimized edge information enhancement texture feature map; and S132, enabling the optimized edge information enhanced texture feature map to pass through an AIGC-based high-resolution image generator to obtain the multi-channel high-resolution edge fusion image. It should be understood that in step S131, the feature distribution optimization is performed on the edge information enhancement texture feature map, and the optimization is performed to adjust the feature distribution so that the edge information can be better highlighted and emphasized, and by optimizing the feature distribution, the contrast and the sharpness of the edge information can be enhanced, so as to improve the visibility and the quality of the edge, and a specific optimization method can be determined according to a specific algorithm and technology, for example, using a method of histogram equalization, contrast enhancement, non-local mean filtering, and the like. In step S132, the optimized edge information enhanced texture feature map is converted into a multi-channel high resolution edge fusion image using a AIGC (Adaptive Image Guided Contextualization) based high resolution image generator. AIGC is an image-guided-based context modeling method that utilizes context information of an input image to improve resolution and quality of the image. By AIGC, edge information and other context information in texture feature maps can be enhanced with optimized edge information, resulting in images with higher resolution and sharper edges. The AIGC method can synthesize more details and textures from a low resolution input image by learning and modeling the context of the image to generate a high resolution edge blending image. That is, the step S131 enhances the quality and contrast of the texture feature map by improving the edge information through feature distribution optimization, and the step S132 converts the optimized texture feature map into a multi-channel high-resolution edge fusion image using an AIGC-based high-resolution image generator. The purpose of these steps is to generate an image with high resolution and sharp edges to improve the visual quality of the image and the visibility of the edge information.
In the technical scheme of the disclosure, the image texture feature map expresses multi-scale local associated image texture semantic features of the low-resolution image, so that after the image texture feature map is weighted by taking the edge weight vector as a weight vector, the edge information enhanced texture feature map still expresses image semantic features enhanced by edge image semantic distribution by taking a feature matrix as a unit, namely, global image semantic feature distribution enhancement of the feature matrix is performed by taking local spatial distribution of partial edge textures of the multi-scale local associated image texture semantic features. In this way, when the edge information enhanced texture feature map is passed through the AIGC-based high-resolution image generator, a scale heuristic distribution probability mapping is performed based on the image semantic feature distribution representation of each feature matrix of the edge information enhanced texture feature map, but considering that the image semantic feature distribution of each feature matrix also includes a hybrid image semantic association feature distribution representation of a local edge-global texture, this may result in a decrease in training efficiency of the AIGC-based high-resolution image generator.
Based on this, the applicant of the present disclosure performs semantic information homogenizing activation of feature rank expression on the edge information enhanced texture feature map when passing the edge information enhanced texture feature map through an AIGC-based high resolution image generator.
Accordingly, in a specific example, performing feature distribution optimization on the edge information enhanced texture feature map to obtain an optimized edge information enhanced texture feature map includes: carrying out feature distribution optimization on the edge information enhancement texture feature map by using the following optimization formula to obtain the optimized edge information enhancement texture feature map; wherein, the optimization formula is:
wherein F is the edge information enhanced texture feature map, F i Is the edge information enhancementThe ith eigenvalue of the texture feature map, log is a 2-base logarithmic function, and α is a weight hyperparameter, f' i Is the i-th feature value of the optimized edge information enhanced texture feature map.
Here, considering that the feature distribution mapping of the edge information enhanced texture feature map V in the high-dimensional feature space to the distribution probability regression space may represent different mapping modes on different feature distribution levels based on the semantic features of the mixed image, so that the mapping policy based on the scale heuristic cannot obtain optimal efficiency, and therefore, the rank expression semantic information based on different norms of feature values is uniform, but not scale is subjected to feature matching, similar feature rank expressions can be activated in a similar manner, and the correlation between feature rank expressions with larger difference can be reduced, so that the problem that the distribution probability expression mapping efficiency of the feature distribution of the edge information enhanced texture feature map F in different spatial rank expressions is low is solved, and the training efficiency of the edge information enhanced texture feature map through the AIGC-based high-resolution image generator is improved.
In summary, according to the multi-channel high-resolution edge fusion method disclosed by the embodiment of the disclosure, edge information can be effectively extracted and fused.
Fig. 6 shows a block diagram of a multi-channel high resolution edge blending system 100 according to an embodiment of the present disclosure. As shown in fig. 6, a multi-channel high resolution edge blending system 100 according to an embodiment of the present disclosure includes: an image acquisition module 110 for acquiring a low resolution image; the enhancement processing module 120 is configured to perform edge information enhancement processing on the low-resolution image to obtain an edge information enhanced texture feature map; and a generating module 130, configured to generate a multi-channel high-resolution edge fusion image based on the edge information enhanced texture feature map.
In one possible implementation, the enhancement processing module 120 includes: the multi-scale feature extraction unit is used for carrying out multi-scale feature extraction on the low-resolution image so as to obtain an image texture feature map; the edge feature extraction unit is used for extracting edge features in the image texture feature map to obtain edge weight vectors; and the weighting unit is used for weighting the image texture feature map by taking the edge weight vector as a weight vector so as to obtain the edge information enhanced texture feature map.
In a possible implementation manner, the multi-scale feature extraction unit is configured to: and passing the low-resolution image through an image texture feature extractor with a multi-scale convolution structure to obtain the image texture feature map.
In a possible implementation manner, the edge feature extraction unit is configured to: performing edge detection on each feature matrix in the image texture feature graph to obtain a sequence of edge feature matrices; and carrying out global averaging pooling on each feature matrix in the sequence of the edge feature matrices to obtain the edge weight vector.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described multi-channel high-resolution edge blending system 100 have been described in detail in the above description of the multi-channel high-resolution edge blending method with reference to fig. 1 to 5, and thus, repetitive descriptions thereof will be omitted.
As described above, the multi-channel high-resolution edge blending system 100 according to the embodiment of the present disclosure may be implemented in various wireless terminals, such as a server or the like having a multi-channel high-resolution edge blending algorithm. In one possible implementation, the multi-channel high-resolution edge blending system 100 according to embodiments of the present disclosure may be integrated into a wireless terminal as one software module and/or hardware module. For example, the multi-channel high-resolution edge blending system 100 may be a software module in the operating system of the wireless terminal, or may be an application developed for the wireless terminal; of course, the multi-channel high-resolution edge blending system 100 can also be one of many hardware modules of the wireless terminal.
Alternatively, in another example, the multi-channel high resolution edge blending system 100 and the wireless terminal may be separate devices, and the multi-channel high resolution edge blending system 100 may be connected to the wireless terminal through a wired and/or wireless network and transmit the interaction information in a agreed data format.
Fig. 7 illustrates an application scenario diagram of a multi-channel high-resolution edge blending method according to an embodiment of the present disclosure. As shown in fig. 7, in this application scenario, first, a low resolution image (e.g., D illustrated in fig. 7) is acquired, and then, the low resolution image is input to a server (e.g., S illustrated in fig. 7) in which a multi-channel high resolution edge blending algorithm is deployed, wherein the server is capable of processing the low resolution image using the multi-channel high resolution edge blending algorithm to generate a multi-channel high resolution edge blended image.
It should be appreciated that the present disclosure provides a technical solution for a multi-channel high-resolution edge blending method, which can effectively improve the visual quality and detail preservation of an image. Specifically, the scheme comprises the following steps: firstly, up-sampling an input low-resolution image through a multichannel Convolutional Neural Network (CNN) to obtain a group of high-resolution sub-images; then, edge detection is carried out on each sub-image, and edge information of each sub-image is extracted; then, carrying out weighted average on the sub-images by utilizing the edge information to obtain a final high-resolution image; finally, the final high resolution image is post-processed to eliminate possible artifacts and noise. The scheme has the advantages that: on one hand, through the multichannel CNN, the texture and structure of the image can be enhanced by utilizing the information of different scales and characteristics at the same time; on the other hand, through edge fusion, the definition and sharpness of the image can be reserved, and blurring and distortion are avoided. Experimental results show that the scheme is superior to the existing method in various evaluation indexes, and has good visual effect and perception quality.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as a memory including computer program instructions executable by a processing component of an apparatus to perform the above-described method.
The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (6)
1. A multi-channel high resolution edge blending method, comprising:
acquiring a low resolution image;
performing edge information enhancement processing on the low-resolution image to obtain an edge information enhancement texture feature map; and
and generating a multi-channel high-resolution edge fusion image based on the edge information enhanced texture feature map.
2. The multi-channel high-resolution edge blending method according to claim 1, wherein performing edge information enhancement processing on the low-resolution image to obtain an edge information enhanced texture feature map comprises:
performing multi-scale feature extraction on the low-resolution image to obtain an image texture feature map;
extracting edge features in the image texture feature map to obtain edge weight vectors; and
and weighting the image texture feature map by taking the edge weight vector as a weight vector to obtain the edge information enhanced texture feature map.
3. The multi-channel high-resolution edge blending method according to claim 2, wherein performing multi-scale feature extraction on the low-resolution image to obtain an image texture feature map comprises:
and passing the low-resolution image through an image texture feature extractor with a multi-scale convolution structure to obtain the image texture feature map.
4. A multi-channel high resolution edge blending method according to claim 3, wherein extracting edge features in said image texture feature map to obtain an edge weight vector comprises:
performing edge detection on each feature matrix in the image texture feature graph to obtain a sequence of edge feature matrices; and
and carrying out global mean pooling on each feature matrix in the sequence of the edge feature matrix to obtain the edge weight vector.
5. The multi-channel high-resolution edge blending method according to claim 4, wherein generating a multi-channel high-resolution edge blended image based on the edge information enhanced texture feature map comprises:
performing feature distribution optimization on the edge information enhancement texture feature map to obtain an optimized edge information enhancement texture feature map; and
and the optimized edge information enhanced texture feature image passes through an AIGC-based high-resolution image generator to obtain the multi-channel high-resolution edge fusion image.
6. The multi-channel high-resolution edge blending method according to claim 5, wherein performing feature distribution optimization on the edge information enhancement texture feature map to obtain an optimized edge information enhancement texture feature map comprises:
carrying out feature distribution optimization on the edge information enhancement texture feature map by using the following optimization formula to obtain the optimized edge information enhancement texture feature map;
wherein, the optimization formula is:
wherein F is the edge information enhanced texture feature map, F i Is the ith eigenvalue of the edge information enhanced texture feature graph, log is a logarithmic function based on 2, alpha is a weight super-parameter, f i ' is the i-th feature value of the optimized edge information enhanced texture feature map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311113762.8A CN117115465A (en) | 2023-08-30 | 2023-08-30 | Multi-channel high-resolution edge fusion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311113762.8A CN117115465A (en) | 2023-08-30 | 2023-08-30 | Multi-channel high-resolution edge fusion method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117115465A true CN117115465A (en) | 2023-11-24 |
Family
ID=88808908
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311113762.8A Pending CN117115465A (en) | 2023-08-30 | 2023-08-30 | Multi-channel high-resolution edge fusion method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117115465A (en) |
-
2023
- 2023-08-30 CN CN202311113762.8A patent/CN117115465A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106934397B (en) | Image processing method and device and electronic equipment | |
| Zhang et al. | Deep image deblurring: A survey | |
| Zhang et al. | Multi-scale single image dehazing using perceptual pyramid deep network | |
| KR102172234B1 (en) | Image processing method and apparatus, and electronic device | |
| EP3948764B1 (en) | Method and apparatus for training neural network model for enhancing image detail | |
| CN110163237B (en) | Model training and image processing method, device, medium and electronic equipment | |
| Saladi et al. | Analysis of denoising filters on MRI brain images | |
| US10198801B2 (en) | Image enhancement using self-examples and external examples | |
| CN111275626A (en) | Video deblurring method, device and equipment based on ambiguity | |
| CN110136055B (en) | Super resolution method and device for image, storage medium and electronic device | |
| Lu et al. | Deep texture and structure aware filtering network for image smoothing | |
| WO2022134971A1 (en) | Noise reduction model training method and related apparatus | |
| CN112602088B (en) | Method, system and computer readable medium for improving quality of low light images | |
| CN113674146A (en) | Image super-resolution | |
| Lau et al. | Variational models for joint subsampling and reconstruction of turbulence-degraded images | |
| CN111951172A (en) | Image optimization method, device, equipment and storage medium | |
| CN111353955A (en) | Image processing method, device, equipment and storage medium | |
| CN116547694A (en) | Method and system for deblurring blurred images | |
| Byun et al. | BitNet: Learning-based bit-depth expansion | |
| Cui et al. | Exploring resolution and degradation clues as self-supervised signal for low quality object detection | |
| Elwarfalli et al. | Fifnet: A convolutional neural network for motion-based multiframe super-resolution using fusion of interpolated frames | |
| Askari Javaran et al. | [Retracted] Using a Blur Metric to Estimate Linear Motion Blur Parameters | |
| Alam et al. | Space-variant blur kernel estimation and image deblurring through kernel clustering | |
| Wali et al. | Recent progress in digital image restoration techniques: a review | |
| Tang et al. | Structure-embedded ghosting artifact suppression network for high dynamic range image reconstruction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |