WO2022166298A1 - 一种图像处理方法、装置、电子设备及可读存储介质 - Google Patents
一种图像处理方法、装置、电子设备及可读存储介质 Download PDFInfo
- Publication number
- WO2022166298A1 WO2022166298A1 PCT/CN2021/130805 CN2021130805W WO2022166298A1 WO 2022166298 A1 WO2022166298 A1 WO 2022166298A1 CN 2021130805 W CN2021130805 W CN 2021130805W WO 2022166298 A1 WO2022166298 A1 WO 2022166298A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- layer
- network
- matrix
- processed
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 239000011159 matrix material Substances 0.000 claims abstract description 127
- 238000012545 processing Methods 0.000 claims abstract description 67
- 238000000034 method Methods 0.000 claims abstract description 26
- 239000013598 vector Substances 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims description 50
- 230000006870 function Effects 0.000 claims description 41
- 238000007906 compression Methods 0.000 claims description 27
- 230000006835 compression Effects 0.000 claims description 25
- 238000007781 pre-processing Methods 0.000 claims description 14
- 230000008521 reorganization Effects 0.000 claims description 14
- 238000005070 sampling Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 10
- 230000006798 recombination Effects 0.000 claims description 6
- 238000005215 recombination Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 230000003247 decreasing effect Effects 0.000 claims description 2
- 230000000717 retained effect Effects 0.000 abstract 1
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 8
- 238000011176 pooling Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- KLDZYURQCUYZBL-UHFFFAOYSA-N 2-[3-[(2-hydroxyphenyl)methylideneamino]propyliminomethyl]phenol Chemical compound OC1=CC=CC=C1C=NCCCN=CC1=CC=CC=C1O KLDZYURQCUYZBL-UHFFFAOYSA-N 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 201000001098 delayed sleep phase syndrome Diseases 0.000 description 1
- 208000033921 delayed sleep phase type circadian rhythm sleep disease Diseases 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009469 supplementation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Definitions
- the present application relates to the technical field of image processing, and in particular, to an image processing method, an image processing apparatus, an electronic device, and a computer-readable storage medium.
- the purpose of the present application is to provide an image processing method, an image processing device, an electronic device and a computer-readable storage medium, through two-dimensional compression, each element in the obtained compressed data only contains the information of the pixels in a part of the image , does not include the information of other pixels other than this part of the image, so the obtained compressed data can retain the spatial information of the image, and a clearer image can be obtained after image reconstruction.
- an image processing method including:
- the image to be processed is left-multiplied by using a row ladder observation matrix to obtain a first matrix;
- the row ladder observation matrix is composed of target non-zero elements and zero elements, and each row vector has two adjacent target non-zero elements. zero elements, the positions of the target non-zero elements in each of the row vectors are different;
- the first matrix is right-multiplied by using the transposed matrix of the row ladder observation matrix to obtain compressed data.
- the application also provides an image processing device, comprising:
- the acquisition module is used to acquire the image to be processed
- the first compression module is used to perform left multiplication processing on the to-be-processed image by using a row ladder observation matrix to obtain a first matrix;
- the row ladder observation matrix is composed of target non-zero elements and zero elements, and each row vector has two the adjacent target non-zero elements, the positions of the target non-zero elements in each of the row vectors are different;
- the second compression module is configured to perform right multiplication processing on the first matrix by using the transposed matrix of the row ladder observation matrix to obtain compressed data.
- the application also provides an electronic device, including a memory and a processor, wherein:
- the memory for storing computer programs
- the processor is configured to execute the computer program to implement the above-mentioned image processing method.
- the present application also provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned image processing method when executed by a processor.
- the image processing method provided by the present application obtains an image to be processed; performs left multiplication processing on the image to be processed by using a row ladder observation matrix to obtain a first matrix;
- the row ladder observation matrix is composed of target non-zero elements and zero elements, and each row vector has two There are two adjacent target non-zero elements, and the positions of the target non-zero elements in each row vector are different;
- the first matrix is multiplied by the transposed matrix of the row ladder observation matrix to obtain the compressed data.
- this method uses the row ladder observation matrix to compress the image to be processed two-dimensionally.
- the row echelon observation matrix is a special matrix, which is an echelon matrix, and has only two elements: zero element and target non-zero element, and each row vector has two target non-zero elements and the two target non-zero elements are adjacent.
- the image to be processed can be left-multiplied by the row ladder observation matrix, and the information in the first dimension can be extracted, and the first matrix can be right-multiplied by its transposed matrix, which can be processed in the second dimension.
- Information extraction completes two-dimensional compression of the image to be processed.
- each element in the obtained compressed data only contains the information of the pixels in the part of the image, and does not include the information of other pixels other than the part of the image, so the obtained compressed data can retain the spatial information of the image.
- a clearer image can be obtained, which solves the problem that the compressed data in the related art has more information loss, which makes the image quality obtained after the image reconstruction is poor.
- the present application also provides an image processing apparatus, an electronic device, and a computer-readable storage medium, which also have the above beneficial effects.
- FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present application
- Fig. 2 is a kind of image compression and reconstruction flow chart provided by the embodiment of this application;
- FIG. 3 is a specific structural diagram of a reconstructed network provided by an embodiment of the present application.
- FIG. 4 is another specific structural diagram of a reconstructed network provided by an embodiment of the present application.
- FIG. 5 is a specific to-be-processed image provided by an embodiment of the present application.
- FIG. 6 is a specific reconstructed image provided by an embodiment of the present application.
- FIG. 7 is another specific reconstructed image provided by an embodiment of the present application.
- FIG. 8 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
- FIG. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
- FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present application.
- the method includes:
- S101 Acquire an image to be processed.
- Each part or all of the steps in the embodiments of the present application may be performed by a designated electronic device, and the number of the electronic devices may be one or more, that is, the image processing may be completed by the cooperation of multiple electronic devices.
- the electronic device may be a server, a computer, an intelligent terminal, etc., which is not limited.
- the types of the electronic devices may be the same or different, and they may communicate through a wired network or a wireless network.
- the image to be processed may be any image, and the size and content thereof are not limited.
- the image to be processed can be input from the outside, for example, the image to be processed can be acquired by an image acquisition device or an image acquisition component built in the electronic device; or an image to be processed sent by other electronic devices can be captured.
- the number of images to be processed can be one or more, for example, any acquired image can be regarded as the image to be processed, or the image to be processed can be determined from several images according to the specified instruction of the image to be processed, for example, some images need to be kept optimal. image quality, do not compress it, some images need to avoid taking up too much storage space, so they are determined as images to be processed.
- the instruction to specify the image to be processed can be input by the user, and can be obtained together with the image to be processed.
- S102 Perform left multiplication processing on the image to be processed by using the row ladder observation matrix to obtain a first matrix.
- the row ladder observation matrix is a special observation matrix, which consists of target non-zero elements and zero elements, each row vector has two adjacent target non-zero elements, and the position of the target non-zero elements in each row vector is different. Specifically, any non-zero value can be selected as the target non-zero element, for example, it can be 1.
- a is used to represent the target non-zero element
- the row ladder observation matrix has multiple row vectors, each of which is The row vector has two adjacent target non-zero elements, so it can be determined that there is no all-zero row vector in the row echelon observation matrix.
- each column vector in the row ladder observation matrix only includes one target non-zero element . Therefore, it can be determined that the row ladder observation matrix in this embodiment is in the following form:
- ⁇ represents the row-echelon observation matrix.
- the special form of the row ladder observation matrix can be used to compress the image to be processed two-dimensionally, so that the compressed data obtained by the compression can have the spatial information corresponding to the image to be processed.
- the image to be processed is regarded as a matrix
- the elements of the matrix are the pixel values of each pixel in the image to be processed
- the row ladder observation matrix is used to left-multiply the image to be processed to obtain a first matrix
- the first matrix is the image to be processed. The result of compressing from the first dimension.
- this embodiment does not limit the specific process of performing left multiplication of the image to be processed to obtain the first matrix.
- the row echelon observation matrix has a certain order, which can compress the part of the image that matches its order, and the size of the image to be processed may not match the row echelon observation matrix.
- various preprocessing such as splitting and supplementation can be performed on the image to be processed, and left multiplication processing is performed after the preprocessing to obtain one or more first matrices.
- S103 Perform right multiplication processing on the first matrix by using the transposed matrix of the row ladder observation matrix to obtain compressed data.
- each element in the obtained compressed data only includes the information of the pixels in a certain part of the image to be processed, and does not include the information of other pixels outside the part, which makes the compressed data retain the space of the image information.
- the transposed matrix of ⁇ is ⁇ T .
- x can be used to represent the image to be processed
- y to represent the compressed data
- the row ladder observation matrix is a 32-order matrix
- the size of the to-be-processed image is 32*32 (pixels)
- the row ladder observation matrix is 32*32 (pixels).
- the target non-zero element is 1.
- the compressed data generation process is:
- each element in the compressed data y includes the information of a certain part of the pixels in the image to be processed, but does not include the information of other parts, and the positional relationship between each element and the information included in the image to be processed corresponds to The positional relationship between the pixels is the same, so the compressed data retains the spatial information of the image to be processed, that is, the spatial correlation information.
- the spatial correlation information between adjacent pixels can be used, and the interference of distant pixels to the current local adjacent pixels can also be excluded, and a clear image can be reconstructed in the image reconstruction stage.
- the image processing method provided by the embodiment of the present application is applied, and a row ladder observation matrix is used to perform two-dimensional compression on the image to be processed.
- the row echelon observation matrix is a special matrix, which is an echelon matrix, and has only two elements: zero element and target non-zero element, and each row vector has two target non-zero elements and the two target non-zero elements are adjacent.
- the row ladder observation matrix While using the row ladder observation matrix to multiply the image to be processed to the left, the information in the first dimension can be extracted, and using its transposed matrix to right-multiply the first matrix, it can be processed in the second dimension.
- Information extraction completes two-dimensional compression of the image to be processed.
- each element in the obtained compressed data only contains the information of the pixels in the part of the image, and does not include the information of other pixels other than the part of the image, so the obtained compressed data can retain the spatial information of the image.
- a clearer image can be obtained, which solves the problem that the compressed data in the related art has more information loss, which makes the image quality obtained after the image reconstruction is poor.
- the size of the image to be processed is larger than the order of the row ladder observation matrix.
- the image to be processed may be divided into multiple parts and compressed respectively.
- the steps of obtaining the first matrix by performing left-multiplication processing on the image to be processed by using the row ladder observation matrix may include:
- Step 11 Split the image to be processed according to the order of the row ladder observation matrix to obtain several sub-images to be processed.
- Step 12 Perform left multiplication processing on each sub-image to be processed by using the row ladder observation matrix to obtain several first matrices.
- the order of the row echelon observation matrix determines the size of the image area that can be processed by the row echelon observation matrix. Therefore, when the size of the image to be processed exceeds the size that can be processed by the row echelon matrix, the image to be processed can be processed according to the order of the row echelon matrix. Split to obtain several sub-images to be processed. The specific method of splitting is not limited in this embodiment. For example, starting from the upper left corner of the image to be processed, the image to be processed can be divided into multiple square images according to the order of the row ladder observation matrix as the step size.
- each sub-image to be processed is left-multiplied by using the row echelon matrix, so as to obtain a plurality of corresponding first matrices.
- using the transposed matrix of the row ladder observation matrix to right-multiply the first matrix to obtain compressed data may include:
- Step 13 Use the transposed matrix to perform right multiplication processing on each of the first matrices to obtain several sub-compressed data.
- Step 14 Splicing the sub-compressed data to obtain compressed data.
- each first matrix is right-multiplied by the transposed matrix to obtain corresponding sub-compressed data, and compressed data can be obtained by splicing the sub-compressed data.
- images of any size to be processed can be processed.
- FIG. 2 is a flowchart of image compression and reconstruction provided by an embodiment of the present application.
- the size of the image to be processed is 256*256 (pixels), it is divided into multiple sub-images of 32*32 (pixels) to be processed, and the corresponding sub-compressed data can be obtained after compressing them.
- the compressed data can be obtained by splicing the data. In the follow-up, the compressed data can be directly input into the reconstruction network, and the corresponding reconstructed image can be obtained.
- the compressed data can be input into the trained reconstruction model, and the reconstruction model is used to perform image reconstruction.
- the reconstruction model usually adopts the fully connected layer to perform the first processing step of the reconstruction process, that is, the fully connected layer is used as the first processing layer of the reconstruction model.
- the fully connected layer has more parameters, so its training process is longer and requires more computing resources.
- the fully connected layer can only reconstruct each part of the image separately, so there is a block effect.
- an upsampling processing layer can be used to replace the fully connected layer to complete image reconstruction. Specifically, it can also include:
- Step 21 Generate a reconstructed network based on the initial network.
- Step 22 Input the compressed data into the reconstruction network to obtain a reconstructed image.
- Step 23 Output the reconstructed image.
- the initial network is a convolutional neural network
- the initial network uses an upsampling processing layer to replace the fully connected layer, that is, the first processing layer of the reconstruction network is an upsampling processing layer.
- the upsampling processing layer may be a basic upsampling layer, or may be other network layers obtained based on the upsampling layer, such as a network layer group formed by combining a preprocessing layer and a pixel reshuffler layer.
- PixelShuffle is a special upsampling method, which can effectively enlarge the reduced feature map.
- the upsampling processing layer has fewer parameters, requires less time and computing resources for training, and at the same time has no limit to the size of the input data, and can reconstruct the entire image to eliminate blockiness.
- the data form of the compressed data is similar to the form of the data processed by the average pooling layer.
- the pooling layer is also called undersampling or downsampling.
- the pooling layer is a type of pooling layer. Therefore, the upsampling processing layer can be used to replace the fully connected layer to process the compressed data to complete the reconstruction of the image.
- FIG. 3 is a specific reconstruction network structure diagram provided by an embodiment of the present application, wherein the upsampling processing layer is the upsampling layer upsampling: UpSampling2D, which is located after the input layer input: InputLayer.
- FIG. 4 is another specific reconstruction network structure diagram provided by an embodiment of the present application, wherein the upsampling processing layer includes a preprocessing layer conv_0 and a pixel reorganization layer pixelsshuffler: PixelShuffler.
- FIG. 5 is a specific image to be processed provided by an embodiment of the application
- FIG. 6 is a specific reconstructed image provided by an embodiment of the application
- FIG. 7 is the application Another specific reconstructed image provided by the embodiment.
- Figure 6 is based on a reconstruction network with fully connected layers
- Figure 7 is based on a reconstruction network with upsampling layers. It can be seen from the partial enlarged image in Figure 6 that after reconstruction by the reconstruction network of the fully connected layer, the obtained image is rougher than the corresponding partial enlarged image in Figure 5, and the reconstruction effect is poor. It can be seen from the partial enlarged image in Fig.
- the first column is the name of each image
- the last row is the average value of PSNR corresponding to each image.
- the image reconstructed by the reconstruction network using the fully connected layer as the first processing layer, the corresponding PSNR values are all smaller than the image obtained by using the upsampling layer or the network layer group composed of the preprocessing layer and the pixel recombination layer as the first processing layer.
- the value of PSNR of the image obtained after reconstruction by the reconstruction network is decibel, and the larger the value, the less distortion of the image. Therefore, it can be seen that the quality of the reconstructed image obtained by the reconstruction network with the upsampling processing layer is better.
- the generation process of the reconstructed network includes:
- Step 31 Obtain a training set, and obtain target training images from the training set.
- Step 32 Input the target training image into the initial network, obtain the output result, and use the output result to calculate the loss value based on the first loss function.
- Step 33 if the decreasing range of the loss value is lower than the preset threshold, replace the first loss function with the second loss function.
- Step 34 Adjust the network parameters of the initial network according to the loss value, and update the target training image until the reconstructed network is obtained.
- the initial network is an untrained reconstruction network, which is a reconstructed network after training.
- some training images are obtained from the training set as target training images, and they are input into the initial network to obtain the output results.
- the first loss function is used to calculate the loss value, and the network parameters are adjusted according to the loss function, and the target training image is updated for iterative training.
- the second loss function can be used again to calculate the loss value, and the iterative training can be continued until the initial network training is completed, and the reconstructed network is obtained.
- This embodiment does not limit the specific types of the first loss function and the second loss function.
- the first loss function is the mean absolute error loss function (ie, the Mean Absolute Deviation function)
- the second loss function is Mean Squared Error loss function (Mean Squared Error function).
- the step of inputting the target training image into the initial network, and obtaining the output result may include:
- Step 41 Upsampling the target training image by using the upsampling layer after the input layer in the reconstruction network to obtain upsampled data;
- Step 42 Input the up-sampling data into a subsequent network layer after the up-sampling layer to obtain the output data.
- the subsequent network layers include multiple network layer groups, and each network layer group is composed of a convolution layer and an activation function layer.
- the upsampling layer is used as the first processing layer, and the target training image does not need to be preprocessed, but can be directly upsampled. After the upsampled data is obtained, subsequent processing is performed on it, and finally the output data is obtained.
- the upsampling processing layer includes a preprocessing layer and a pixel reorganization layer
- the target training image is input into the initial network, and the steps of obtaining the output result may include:
- Step 51 Preprocess the target training image by using the preprocessing layer after the input layer in the reconstruction network to obtain preprocessed data.
- Step 52 Perform pixel reorganization processing on the preprocessed data by using the pixel reorganization layer to obtain reorganized data.
- Step 53 Input the recombined data into the subsequent network layer after the pixel recombination layer to obtain output data.
- the main function of the pixel reorganization layer is to convert the low-resolution feature map to a high-resolution feature map through convolution and multi-channel recombination.
- the whole process can be equivalent to convolution first, and then periodic pixel selection. . Therefore, in this embodiment, a convolutional layer may be set before the pixel reorganization layer and after the input layer, and the convolutional layer is determined as a preprocessing layer, which is used to preprocess the input image for subsequent pixel reorganization.
- the subsequent network layers include multiple network layer groups, and each network layer group is composed of a convolution layer and an activation function layer.
- the number of network layer groups is not limited, for example, it can be three.
- the output interval of the activation function layer in the last network layer group can be limited, so as to avoid the effect of pixel value overflow on the effect of the reconstructed image.
- the last network layer in the subsequent network layer The activation function layer of the layer group is the output layer.
- the output interval of the output layer is set as a preset interval. The specific size of the preset interval is not limited. To 0-1, the pixel value of each pixel in the reconstructed image is between 0-255.
- FIG. 8 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application, including:
- an acquisition module 110 configured to acquire an image to be processed
- the first compression module 120 is configured to perform left multiplication processing on the to-be-processed image by using a row ladder observation matrix to obtain a first matrix;
- the row ladder observation matrix is composed of target non-zero elements and zero elements, and each row vector has two elements. adjacent said target non-zero elements, the positions of said target non-zero elements in each of said row vectors are different;
- the second compression module 130 is configured to perform right-multiplication processing on the first matrix by using the transposed matrix of the row ladder observation matrix to obtain compressed data.
- the first compression module 120 includes:
- the splitting unit is used for splitting the to-be-processed image according to the order of the row ladder observation matrix to obtain several to-be-processed sub-images;
- a first compression unit configured to perform left-multiplication processing on each sub-image to be processed by using the row ladder observation matrix to obtain several first matrices
- the second compression module 130 includes:
- the second compression unit is used to perform right multiplication processing on each of the first matrices by using the transposed matrix to obtain several sub-compressed data;
- the splicing unit is used for splicing the sub-compressed data to obtain compressed data.
- a generation module used for generating a reconstruction network based on an initial network
- the initial network is a convolutional neural network
- the initial network uses an upsampling processing layer to replace the fully connected layer
- the input module is used to input the compressed data into the reconstruction network to obtain the reconstructed image
- the output module is used to output the reconstructed image.
- the generation module includes:
- the target training image determination module is used to obtain the training set and obtain the target training image from the training set;
- the loss value calculation module is used to input the target training image into the initial network, obtain the output result, and use the output result to calculate the loss value based on the first loss function;
- a function replacement module configured to replace the first loss function with the second loss function if the drop of the loss value is lower than the preset threshold
- the parameter adjustment module is used to adjust the network parameters of the initial network according to the loss value, and update the target training image until the reconstructed network is obtained.
- the upsampling processing layer is an upsampling layer
- the loss value calculation module includes:
- an up-sampling unit used for up-sampling the target training image by using the up-sampling layer after the input layer in the reconstruction network to obtain up-sampling data
- a first output data generating unit configured to input the up-sampled data into a subsequent network layer after the up-sampling layer to obtain the output data;
- the subsequent network layer includes a plurality of network layer groups, each network layer group It consists of a convolutional layer and an activation function layer.
- the upsampling processing layer includes a preprocessing layer and a pixel recombination layer
- the loss value calculation module includes:
- a preprocessing unit configured to preprocess the target training image by using the preprocessing layer after the input layer in the reconstruction network to obtain preprocessing data
- the reorganization unit is used to perform pixel reorganization processing on the preprocessed data by using the pixel reorganization layer to obtain reorganized data;
- the second output data generation unit is used to input the recombined data into the subsequent network layer after the pixel reorganization layer to obtain output data;
- the subsequent network layer includes a plurality of network layer groups, and each network layer group consists of a convolution layer and an activation function layer composition.
- the activation function layer of the last network layer group in the subsequent network layers is the output layer, and the output interval of the output layer is a preset interval.
- the electronic device provided by the embodiments of the present application will be introduced below, and the electronic device described below and the image processing method described above may refer to each other correspondingly.
- the electronic device 100 may include a processor 101 and a memory 102 , and may further include one or more of a multimedia component 103 , an information input/information output (I/O) interface 104 and a communication component 105 .
- a multimedia component 103 may be included in the electronic device 100 .
- I/O information input/information output
- the processor 101 is used to control the overall operation of the electronic device 100 to complete all or part of the steps in the above-mentioned image processing method;
- the memory 102 is used to store various types of data to support the operation of the electronic device 100. These data For example, instructions for any application or method to operate on the electronic device 100 may be included, as well as application-related data.
- the memory 102 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory) Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read- One or more of Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.
- SRAM Static Random Access Memory
- EEPROM Electrically Erasable Programmable Read-Only Memory
- EPROM Erasable Programmable Read-Only Memory
- PROM Programmable Read-Only Memory
- Read- One or more of Only Memory ROM
- magnetic memory flash memory
- flash memory magnetic disk or optical disk.
- Multimedia components 103 may include screen and audio components.
- the screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals.
- the audio component may include a microphone for receiving external audio signals.
- the received audio signal may be further stored in the memory 102 or transmitted through the communication component 105 .
- the audio assembly also includes at least one speaker for outputting audio signals.
- the I/O interface 104 provides an interface between the processor 101 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse, a button, and the like. These buttons can be virtual buttons or physical buttons.
- the communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC for short), 2G, 3G or 4G, or one or a combination of them, so the corresponding communication component 105 may include: Wi-Fi parts, Bluetooth parts, NFC parts.
- the electronic device 100 may be implemented by one or more Application Specific Integrated Circuit (ASIC for short), Digital Signal Processor (DSP for short), Digital Signal Processing Device (DSPD for short), Programmable logic device (Programmable Logic Device, PLD for short), Field Programmable Gate Array (Field Programmable Gate Array, FPGA for short), controller, microcontroller, microprocessor or other electronic components are implemented for implementing the above embodiments The given image processing method.
- ASIC Application Specific Integrated Circuit
- DSP Digital Signal Processor
- DSPD Digital Signal Processing Device
- PLD Programmable logic device
- Field Programmable Gate Array Field Programmable Gate Array
- controller microcontroller, microprocessor or other electronic components are implemented for implementing the above embodiments The given image processing method.
- the computer-readable storage medium provided by the embodiments of the present application is introduced below, and the computer-readable storage medium described below and the image processing method described above may refer to each other correspondingly.
- the present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.
- the computer-readable storage medium may include: a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc. that can store program codes medium.
- the steps of a method or algorithm described in connection with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two.
- the software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
本申请公开了一种图像处理方法、装置、电子设备及计算机可读存储介质,该方法包括:获取待处理图像;利用行阶梯观测矩阵对待处理图像进行左乘处理,得到第一矩阵;行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的目标非零元素,目标非零元素在各个行向量中的位置不同;利用行阶梯观测矩阵的转置矩阵对第一矩阵进行右乘处理,得到压缩数据;该方法得到的压缩数据能够保留图像的空间信息,在图像重构后可以得到更加清晰的图像。
Description
本申请涉及图像处理技术领域,特别涉及一种图像处理方法、图像处理装置、电子设备及计算机可读存储介质。
发明背景
随着大数据和人工智能的快速发展,用户对图像视频的需求量大大提高,需要大量的存储空间和通信资源来存储和传输图像。为了减少存储资源和通信资源的消耗,通常在存储和传输前对图像进行压缩,并在需要时对图像进行重构。压缩感知(Compressed Sensing,CS)理论以远低于奈奎斯特频率的速度成功实现了对信号的同时采样与压缩。降低在传输、存储过程中的带宽资源浪费和硬件设备代价。相关技术在通常采用观测矩阵对图像进行一维压缩,得到压缩数据,在需要时利用压缩数据进行数据重构,得到图像。然而,相关技术得到的压缩数据具有较多的信息损失,在图像重构后得到的图像质量较差。
因此,如何解决相关技术存在的压缩数据信息损失较多的问题,是本领域技术人员需要解决的技术问题。
发明内容
有鉴于此,本申请的目的在于提供一种图像处理方法、图像处理装置、电子设备及计算机可读存储介质,通过二维压缩,使得得到的压缩数据中各个元素仅包含部分图像中像素的信息,不包括该部分图像以外的其他像素的信息,因此得到的压缩数据能够保留图像的空间信息,在图像重构后可以得到更加清晰的图像。
为解决上述技术问题,本申请提供了一种图像处理方法,包括:
获取待处理图像;
利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵;所述行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的所述目标非零元素,所述目标非零元素在各个所述行向量中的位置不同;
利用所述行阶梯观测矩阵的转置矩阵对所述第一矩阵进行右乘处理,得到压缩数据。
本申请还提供了一种图像处理装置,包括:
获取模块,用于获取待处理图像;
第一压缩模块,用于利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵;所述行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的所述目标非零元素,所述目标非零元素在各个所述行向量中的位置不同;
第二压缩模块,用于利用所述行阶梯观测矩阵的转置矩阵对所述第一矩阵进行右乘处理,得到压缩数据。
本申请还提供了一种电子设备,包括存储器和处理器,其中:
所述存储器,用于保存计算机程序;
所述处理器,用于执行所述计算机程序,以实现上述的图像处理方法。
本申请还提供了一种计算机可读存储介质,用于保存计算机程序,其中,所述计算机程序被处理器执行时实现上述的图像处理方法。
本申请提供的图像处理方法,获取待处理图像;利用行阶梯观测矩阵对待处理图像进行左乘处理,得到第一矩阵;行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的目标非零元素,目标非零元素在各个行向量中的位置不同;利用行阶梯观测矩阵的转置矩阵对第一矩阵进行右乘处理,得到压缩数据。
可见,该方法利用行阶梯观测矩阵对待处理图像进行二维压缩。行阶梯观测矩阵为特殊的矩阵,其为阶梯矩阵,且仅具有零元素和目标非零元素两种元素,各个行向量中具有两个目标非零元素且两个目标非零元素相邻。图片各个局部之间具有一定的空间相关性,一维压缩的过程会造成图片空间信息的损失。而利用行阶梯观测矩阵对待处理图像进行左乘,可以对其进行第一个维度上的信息提取,利用其转置矩阵对第一矩阵进行右乘处理,可以对其进行第二个维度上的信息提取,完成对待处理图像的二维压缩。通过二维压缩,使得得到的压缩数据中各个元素仅包含部分图像中像素的信息,不包括该部分图像以外的其他像素的信息,因此得到的压缩数据能够保留图像的空间信息,在图像重构后可以得到更加清晰的图像,解决了相关技术存在的压缩数据具有较多的信息损失,使得在图像重构后得到的图像质量较差的问题。
此外,本申请还提供了一种图像处理装置、电子设备及计算机可读存储介质,同样具有上述有益效果。
附图简要说明
为了更清楚地说明本申请实施例或相关技术中的技术方案,下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据提供的附图获得其他的附图。
图1为本申请实施例提供的一种图像处理方法流程图;
图2为本申请实施例提供的一种图像压缩和重构流程图;
图3为本申请实施例提供的一种具体的重构网络结构图;
图4为本申请实施例提供的另一种具体的重构网络结构图;
图5为本申请实施例提供的一种具体的待处理图像;
图6为本申请实施例提供的一种具体的重构图像;
图7为本申请实施例提供的另一种具体的重构图像;
图8为本申请实施例提供的一种图像处理装置的结构示意图;
图9为本申请实施例提供的一种电子设备的结构示意图。
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
请参考图1,图1为本申请实施例提供的一种图像处理方法流程图。该方法包括:
S101:获取待处理图像。
本申请实施例中的各个部分或全部步骤可以由指定的电子设备执行,该电子设备的数量可以为一个或多个,即可以由多个电子设备配合完成图像处理。电子设备可以为服务器、计算机、智能终端等类型,对此不做限定。当电子设备的数量为多个时,各个电子设备的类型可以相同也可以不同,其可以通过有线网络或无线网络进行通信。
在本实施例中,待处理图像可以为任意图像,其大小、内容等不做限定。待处理图像可以由外部输入,例如可以通过图像获取设备或电子设备自带的图 像获取部件获取待处理图像;或者可以捕获其他电子设备发送的待处理图像。待处理图像的数量可以为一个或多个,例如可以将获取到的任意图像均作为待处理图像,或者可以根据待处理图像指定指令从若干图像中确定待处理图像,例如有些图像需要保持最佳画质,不对其进行压缩,有些图像需要避免其占用过多的存储空间,因此将其确定为待处理图像。待处理图像指定指令可以由用户输入,可以与待处理图像共同获取,其形式不做限定,例如可以为图像名称或图像序号,即将具有该名称或序号的图像确定为待处理图像。
S102:利用行阶梯观测矩阵对待处理图像进行左乘处理,得到第一矩阵。
行阶梯观测矩阵为一种特殊的观测矩阵,其由目标非零元素和零元素组成,各个行向量具有两个相邻的目标非零元素,目标非零元素在各个行向量中的位置不同。具体的,可以选择任意一个非零值作为目标非零元素,例如可以为1,在本实施例中,利用a对目标非零元素进行表示,行阶梯观测矩阵中具有多个行向量,每个行向量具有两个相邻的目标非零元素,因此可以确定,行阶梯观测矩阵中不存在全零行向量。每个行向量中目标非零元素的位置均不同,即每个行向量中的目标非零元素对应的列序号均不同,因此行阶梯观测矩阵中的每个列向量仅包括一个目标非零元素。因此可以确定,本实施例中的行阶梯观测矩阵呈如下形式:
其中,φ表示行阶梯观测矩阵。行阶梯观测矩阵的特殊形式,可以利用其对待处理图像进行二维压缩,使得压缩得到的压缩数据能够具有待处理图像对应的空间信息。具体的,将待处理图像视为一个矩阵,矩阵的元素为待处理图像中各个像素的像素值,利用行阶梯观测矩阵对待处理图像进行左乘,得到第一矩阵,第一矩阵为对待处理图像从第一个维度进行压缩得到的结果。
需要说明的是,本实施例并不限定对待处理图像进行左乘得到第一矩阵的具体过程。行阶梯观测矩阵具有一定阶数,其能够对与其阶数匹配的图像部分进行压缩,而待处理图像的大小可能无法与行阶梯观测矩阵匹配。在这种情况下,可以对待处理图像进行拆分、补充等各种预处理,在预处理后进行左乘处理,得到一个或多个第一矩阵。
S103:利用行阶梯观测矩阵的转置矩阵对第一矩阵进行右乘处理,得到压缩数据。
在得到第一矩阵后,对行阶梯观测矩阵进行转置处理,得到对应的转置矩阵,并利用转置矩阵对第一矩阵进行右乘处理,实现从第二个维度对待处理图像进行压缩的效果,得到对应的压缩数据。利用转置矩阵进行右乘处理,得到的压缩数据中各个元素仅包括待处理图像某个部分中像素的信息,而不包括该部分以外的其他像素的信息,这使得压缩数据保留了图像的空间信息。
具体的,φ的转置矩阵即为φ
T。在一种具体的实施方式中,可以利用x表示待处理图像,y表示压缩数据,行阶梯观测矩阵为32阶矩阵,待处理图像的大小为32*32(像素),行阶梯观测矩阵中的目标非零元素为1。在这种情况下,压缩数据的生成过程为:
由此可见,压缩数据y中的每个元素都包括了待处理图像中某一部分像素的信息,而不包括其他部分的信息,各个元素之间的位置关系与其包括的信息在待处理图像中对应的像素之间的位置关系相同,因此压缩数据保留了待处理图像的空间信息,即空间相关性信息。在重构时,能利用相邻像素之间的空间相关性信息,也排除了较远像素对当前局部邻近像素的干扰,可以在图像重构阶段重构出清晰的图像。
应用本申请实施例提供的图像处理方法,利用行阶梯观测矩阵对待处理图像进行二维压缩。行阶梯观测矩阵为特殊的矩阵,其为阶梯矩阵,且仅具有零元素和目标非零元素两种元素,各个行向量中具有两个目标非零元素且两个目标非零元素相邻。图片各个局部之间具有一定的空间相关性,一维压缩的过程会造成图片空间信息的损失。而利用行阶梯观测矩阵对待处理图像进行左乘, 可以对其进行第一个维度上的信息提取,利用其转置矩阵对第一矩阵进行右乘处理,可以对其进行第二个维度上的信息提取,完成对待处理图像的二维压缩。通过二维压缩,使得得到的压缩数据中各个元素仅包含部分图像中像素的信息,不包括该部分图像以外的其他像素的信息,因此得到的压缩数据能够保留图像的空间信息,在图像重构后可以得到更加清晰的图像,解决了相关技术存在的压缩数据具有较多的信息损失,使得在图像重构后得到的图像质量较差的问题。
基于上述实施例,本实施例将对上述实施例中的若干步骤进行具体的阐述。在一种可能的实施方式中,待处理图像的大小大于行阶梯观测矩阵的阶数,在这种情况下,可以对待处理图像拆分为多个部分,并分别进行压缩。利用行阶梯观测矩阵对待处理图像进行左乘处理,得到第一矩阵的步骤可以包括:
步骤11:根据行阶梯观测矩阵的阶数对待处理图像进行拆分,得到若干个待处理子图像。
步骤12:利用行阶梯观测矩阵分别对各个待处理子图像进行左乘处理,得到若干个第一矩阵。
行阶梯观测矩阵的阶数决定了行阶梯观测矩阵能够处理的图像区域的大小,因此当待处理图像的大小超过了行阶梯矩阵能够处理的大小时,可以根据行阶梯矩阵的阶数对待处理图像进行拆分,得到若干个待处理子图像。拆分的具体方式本实施例不做限定,例如可以从待处理图像的左上角开始,按照行阶梯观测矩阵的阶数为步长将待处理图像划分为多个正方形图像,若待处理图像的右方和/或下方的部分较小,无法组成正方形图像,则可以对其进行补零处理,补零处理后行阶梯观测矩阵即可对其进行处理。在得到多个待处理子图像后,利用行阶梯矩阵分别对各个待处理子图像进行左乘处理,即可得到多个对应的第一矩阵。
相应的,利用行阶梯观测矩阵的转置矩阵对第一矩阵进行右乘处理,得到压缩数据的步骤可以包括:
步骤13:利用转置矩阵分别对各个第一矩阵进行右乘处理,得到若干个子压缩数据。
步骤14:对子压缩数据进行拼接,得到压缩数据。
在得到第一矩阵后,利用转置矩阵对各个第一矩阵进行右乘处理,即可得到对应的子压缩数据,通过将子压缩数据进行拼接即可得到压缩数据。利用上述处理方式,可以对任意大小的待处理图像进行处理。
请参考图2,图2为本申请实施例提供的一种图像压缩和重构流程图。待处理图像的大小为256*256(像素),将其拆分为多个32*32(像素)的待处理子图像,对其进行压缩后即可得到对应的子压缩数据,通过对子压缩数据进行拼接即可得到压缩数据。在后续,可以直接将压缩数据输入重构网络,即可得到对应的重构图像。
基于上述实施例,得到压缩数据后,在需要进行图像重构时,可以将压缩数据输入训练好的重构模型,利用重构模型进行图像重构。相关技术中,重构模型通常采用全连接层执行重构处理的第一个处理步骤,即将全连接层作为重构模型的第一处理层。然而,全连接层的参数较多,因此其训练过程较长,所需的计算资源也较多。同时,全连接层仅能对图像的各个部分分别进行重构,因此存在块效应。为了解决上述问题,可以采用上采样处理层对全连接层进行替代,完成图像重构。具体的,还可以包括:
步骤21:基于初始网络生成重构网络。
步骤22:将压缩数据输入重构网络,得到重构图像。
步骤23:输出重构图像。
在本实施例中,初始网络为卷积神经网络,初始网络利用上采样处理层替代全连接层,即重构网络的第一处理层为上采样处理层。上采样处理层可以为基本的上采样层,或者可以为基于上采样层得到的其他网络层,例如预处理层和像素重组层(PixelShuffler Layer)组合而成的网络层组。像素重组(PixelShuffle)是一种特殊的上采样方式,其可以对缩小后的特征图进行有效的放大。上采样处理层的参数更少,训练所需的时间和计算资源较少,同时对输入数据的大小没有限制,可以对整张图像进行重构,消除块效应。在利用上述压缩数据生成方式得到压缩数据后,可以发现压缩数据的数据形式与平均池化层对数据处理后的形式了相似,池化(Pooling)层也称为欠采样或下采样,平均池化层是池化层中的一种。因此可以利用上采样处理层替代全连接层对压缩数据进行处理,完成对图像的重构。
本实施例并不限定重构网络的具体结构,例如可以参考图3和图4。图3为本申请实施例提供的一种具体的重构网络结构图,其中上采样处理层即为上采样层upsampling:UpSampling2D,其位于输入层input:InputLayer之后。图4为本申请实施例提供的另一种具体的重构网络结构图,其中上采样处理层包括预处理层conv_0和像素重组层pixelshuffler:PixelShuffler。
请参考图5、图6和图7,图5为本申请实施例提供的一种具体的待处理图像,图6为本申请实施例提供的一种具体的重构图像,图7为本申请实施例提供的另一种具体的重构图像。图6基于具有全连接层的重构网络得到,图7基于具有上采样处理层的重构网络得到。从图6中的局部放大图可以看出,经过全连接层的重构网络进行重构后,得到的图像比图5中对应的局部放大图更加粗糙,重构效果较差。从图7的局部放大图可以看出,经过上采样处理层替换全连接层作为第一处理层的重构网络进行图像重构后,得到的图像更加细腻,与图6中的局部放大图相比,可以确定图7的效果更好,比图6更加清晰。在实际测试中,以PSNR(Peak Signal-to-Noise Ratio,峰值信噪比)为标准,对具有全连接层的重构网络、图3所示的重构网络和图4所示的重构网络进行了测试,效果如表1所示:
表1,效果比对表
表1中第一列为各个图像的名称,最后一行为各个图像对应的PSNR的平均值。利用全连接层作为第一处理层的重构网络重构得到的图像,其对应的PSNR的值均小于利用上采样层或预处理层和像素重组层组成的网络层组作为第一处理层的重构网络重构后得到的图像的PSNR的值。PSNR的单位为分贝,其值越大,表示图像失真越少。因此可以看出,具有上采样处理层的重构网络得到的重构图像的质量更好。
可以理解的是,在使用重构网络得到重构图像前,需要对其进行训练。为了保证重构网络的训练效率,可以采用多个的损失函数分阶段进行训练。具体的,重构网络的生成过程,包括:
步骤31:获取训练集,并从训练集中获取目标训练图像。
步骤32:将目标训练图像输入初始网络,得到输出结果,并利用输出结果基于第一损失函数计算损失值。
步骤33:若损失值的下降幅度低于预设阈值,则将第一损失函数替换为第二损失函数。
步骤34:根据损失值调节初始网络的网络参数,并更新目标训练图像,直至得到重构网络。
初始网络为未经过训练的重构网络,其经过训练即为重构网络。在进行训练时,从训练集中获取部分训练图像作为目标训练图像,将其输入初始网络得到输出结果。在训练的初始阶段,先用第一损失函数计算损失值,并根据损失函数调整网络参数,并更新目标训练图像,进行迭代训练。每次在获取到损失值后,利用其与上一次损失值进行比较,判断损失值的下降幅度是否低于预设阈值,若损失值下降幅度低于预设阈值,说明模型训练效果不佳,因此可以利用第二损失函数替换第一损失函数,并继续进行训练,重新利用第二损失函数计算损失值,继续进行迭代训练,直至初始网络训练完毕,得到重构网络。本实施例并不限定第一损失函数与第二损失函数的具体类型,例如在一种实施方式中,第一损失函数为平均绝对误差损失函数(即Mean Absolute Deviation函数),第二损失函数为均方误差损失函数(Mean Squared Error函数)。
基于上述实施例,在一种实施方式中,当上采样处理层为上采样层时,将目标训练图像输入初始网络,得到输出结果的步骤可以包括:
步骤41:利用所述重构网络中输入层后的上采样层对所述目标训练图像进行上采样,得到上采样数据;
步骤42:将所述上采样数据输入所述上采样层后的后续网络层,得到所述输出数据。
本实施例中所述后续网络层包括多个网络层组,每个网络层组由一个卷积层和一个激活函数层组成。本实施例中利用上采样层作为第一处理层,无需对目标训练图像进行预处理,直接对其进行上采样即可,在得到上采样数据后对其进行后续处理,最后得到输出数据。
基于上述实施例,在另一种实施方式中,当上采样处理层包括预处理层和像素重组层时,将目标训练图像输入初始网络,得到输出结果的步骤可以包括:
步骤51:利用重构网络中输入层后的预处理层对目标训练图像进行预处理,得到预处理数据。
步骤52:利用像素重组层对预处理数据进行像素重组处理,得到重组数据。
步骤53:将重组数据输入像素重组层后的后续网络层,得到输出数据。
像素重组层的主要功能是将低分辨的特征图,通过卷积和多通道间的重组得到高分辨率的特征图,整个处理过程可以等价为先进行卷积,再进行周期性的像素选择。因此在本实施例中,可以在像素重组层之前、输入层之后设置卷积层,将该卷积层确定为预处理层,利用其对输入的图像进行预处理,以便后续进行像素重组。因此在目标训练图像输入后,先利用卷积层对其进行预处理,得到预处理数据,再利用像素重组成得到重组数据,最后利用后序网络层对重组数据进行处理,得到输出数据。本实施例中,后续网络层包括多个网络层组,每个网络层组由一个卷积层和一个激活函数层组成。网络层组的数量不做限定,例如可以为3个。
进一步的,为了避免像素值溢出,可以对最后一个网络层组中的激活函数层的输出区间进行限定,避免像素值溢出对重构图像的效果造成影响,具体的,后续网络层中最后一个网络层组的激活函数层即为输出层,输出层的输出区间被设定为预设区间,预设区间的具体大小不做限定,例如可以为0~1,即将像素值0~255归一化至0~1,使重构得到的图像中各个像素的像素值均处于0~255之间。
下面对本申请实施例提供的图像处理装置进行介绍,下文描述的图像处理装置与上文描述的图像处理方法可相互对应参照。
请参考图8,图8为本申请实施例提供的一种图像处理装置的结构示意图,包括:
获取模块110,用于获取待处理图像;
第一压缩模块120,用于利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵;所述行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的所述目标非零元素,所述目标非零元素在各个所述行向量中的位置不同;
第二压缩模块130,用于利用行阶梯观测矩阵的转置矩阵对第一矩阵进行右乘处理,得到压缩数据。
可选地,第一压缩模块120,包括:
拆分单元,用于根据行阶梯观测矩阵的阶数对待处理图像进行拆分,得到若干个待处理子图像;
第一压缩单元,用于利用行阶梯观测矩阵分别对各个待处理子图像进行左乘处理,得到若干个第一矩阵;
相应的,第二压缩模块130,包括:
第二压缩单元,用于利用转置矩阵分别对各个第一矩阵进行右乘处理,得到若干个子压缩数据;
拼接单元,用于对子压缩数据进行拼接,得到压缩数据。
可选地,还包括:
生成模块,用于基于初始网络生成重构网络;所述初始网络为卷积神经网络,所述初始网络利用上采样处理层替代全连接层;
输入模块,用于将压缩数据输入重构网络,得到重构图像;
输出模块,用于输出重构图像。
可选地,生成模块包括:
目标训练图像确定模块,用于获取训练集,并从训练集中获取目标训练图像;
损失值计算模块,用于将目标训练图像输入初始网络,得到输出结果,并利用输出结果基于第一损失函数计算损失值;
函数替换模块,用于若损失值的下降幅度低于预设阈值,则将第一损失函数替换为第二损失函数;
参数调节模块,用于根据损失值调节初始网络的网络参数,并更新目标训练图像,直至得到重构网络。
可选地,所述上采样处理层为上采样层时;
相应的,损失值计算模块,包括:
上采样单元,用于利用所述重构网络中输入层后的上采样层对所述目标训练图像进行上采样,得到上采样数据;
第一输出数据生成单元,用于将所述上采样数据输入所述上采样层后的后续网络层,得到所述输出数据;所述后续网络层包括多个网络层组,每个网络层组由一个卷积层和一个激活函数层组成。
可选地,上采样处理层包括预处理层和像素重组层时;
相应的,损失值计算模块,包括:
预处理单元,用于利用重构网络中输入层后的所述预处理层对目标训练图像进行预处理,得到预处理数据;
重组单元,用于利用像素重组层对预处理数据进行像素重组处理,得到重组数据;
第二输出数据生成单元,用于将重组数据输入像素重组层后的后续网络层,得到输出数据;后续网络层包括多个网络层组,每个网络层组由一个卷积层和一个激活函数层组成。
可选地,后续网络层中最后一个网络层组的激活函数层为输出层,输出层的输出区间为预设区间。
下面对本申请实施例提供的电子设备进行介绍,下文描述的电子设备与上文描述的图像处理方法可相互对应参照。
请参考图9,图9为本申请实施例提供的一种电子设备的结构示意图。其中电子设备100可以包括处理器101和存储器102,还可以进一步包括多媒体组件103、信息输入/信息输出(I/O)接口104以及通信组件105中的一种或多种。
其中,处理器101用于控制电子设备100的整体操作,以完成上述的图像处理方法中的全部或部分步骤;存储器102用于存储各种类型的数据以支持在电子设备100的操作,这些数据例如可以包括用于在该电子设备100上操作的任何应用程序或方法的指令,以及应用程序相关的数据。该存储器102可以由任何类型的易失性或非易失性存储设备或者它们的组合实现,例如静态随机存取存储器 (Static Random Access Memory,SRAM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory,EEPROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory,EPROM)、可编程只读存储器(Programmable Read-Only Memory,PROM)、只读存储器(Read-Only Memory,ROM)、磁存储器、快闪存储器、磁盘或光盘中的一种或多种。
多媒体组件103可以包括屏幕和音频组件。其中屏幕例如可以是触摸屏,音频组件用于输出和/或输入音频信号。例如,音频组件可以包括一个麦克风,麦克风用于接收外部音频信号。所接收的音频信号可以被进一步存储在存储器102或通过通信组件105发送。音频组件还包括至少一个扬声器,用于输出音频信号。I/O接口104为处理器101和其他接口模块之间提供接口,上述其他接口模块可以是键盘,鼠标,按钮等。这些按钮可以是虚拟按钮或者实体按钮。通信组件105用于电子设备100与其他设备之间进行有线或无线通信。无线通信,例如Wi-Fi,蓝牙,近场通信(Near Field Communication,简称NFC),2G、3G或4G,或它们中的一种或几种的组合,因此相应的该通信组件105可以包括:Wi-Fi部件,蓝牙部件,NFC部件。
电子设备100可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit,简称ASIC)、数字信号处理器(Digital Signal Processor,简称DSP)、数字信号处理设备(Digital Signal Processing Device,简称DSPD)、可编程逻辑器件(Programmable Logic Device,简称PLD)、现场可编程门阵列(Field Programmable Gate Array,简称FPGA)、控制器、微控制器、微处理器或其他电子元件实现,用于执行上述实施例给出的图像处理方法。
下面对本申请实施例提供的计算机可读存储介质进行介绍,下文描述的计算机可读存储介质与上文描述的图像处理方法可相互对应参照。
本申请还提供一种计算机可读存储介质,计算机可读存储介质上存储有计算机程序,计算机程序被处理器执行时实现上述的图像处理方法的步骤。
该计算机可读存储介质可以包括:U盘、移动硬盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本说明书中各个实施例采用递进的方式描述,每个实施例重点说明的都是与其它实施例的不同之处,各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言,由于其与实施例公开的方法相对应,所以描述的比较简单,相关之处参见方法部分说明即可。
本领域技术人员还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件的方式来执行,取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应该认为超出本申请的范围。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块,或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
最后,还需要说明的是,在本文中,诸如第一和第二等之类的关系属于仅仅用来将一个实体或者操作与另一个实体或者操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语包括、包含或者其他任何变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。
本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想;同时,对于本领域的一般技术人员,依据本申请的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本申请的限制。
Claims (13)
- 一种图像处理方法,其中,包括:获取待处理图像;利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵;所述行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的所述目标非零元素,所述目标非零元素在各个所述行向量中的位置不同;利用所述行阶梯观测矩阵的转置矩阵对所述第一矩阵进行右乘处理,得到压缩数据。
- 根据权利要求1所述的图像处理方法,其中,所述利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵,包括:根据所述行阶梯观测矩阵的阶数对所述待处理图像进行拆分,得到若干个待处理子图像;利用所述行阶梯观测矩阵分别对各个所述待处理子图像进行左乘处理,得到若干个第一矩阵;相应的,所述利用所述行阶梯观测矩阵的转置矩阵对所述第一矩阵进行右乘处理,得到所述压缩数据,包括:利用所述转置矩阵分别对各个所述第一矩阵进行右乘处理,得到若干个子压缩数据;对所述子压缩数据进行拼接,得到所述压缩数据。
- 根据权利要求1所述的图像处理方法,其中,还包括:基于初始网络生成重构网络;所述初始网络为卷积神经网络,所述初始网络利用上采样处理层替代全连接层;将所述压缩数据输入所述重构网络,得到重构图像;输出所述重构图像。
- 根据权利要求3所述的图像处理方法,其中,所述重构网络的生成过程,包括:获取训练集,并从所述训练集中获取目标训练图像;将所述目标训练图像输入初始网络,得到输出结果,并利用所述输出结果基于第一损失函数计算损失值;若所述损失值的下降幅度低于预设阈值,则将所述第一损失函数替换为第二损失函数;根据所述损失值调节所述初始网络的网络参数,并更新所述目标训练图像,直至得到所述重构网络。
- 根据权利要求4所述的图像处理方法,其中,所述上采样处理层为上采样层;相应的,所述将所述目标训练图像输入初始网络,得到输出结果,包括:利用所述重构网络中输入层后的上采样层对所述目标训练图像进行上采样,得到上采样数据;将所述上采样数据输入所述上采样层后的后续网络层,得到所述输出数据;所述后续网络层包括多个网络层组,每个网络层组由一个卷积层和一个激活函数层组成。
- 根据权利要求4所述的图像处理方法,其中,所述上采样处理层包括预处理层和像素重组层;相应的,所述将所述目标训练图像输入初始网络,得到输出结果,包括:利用所述重构网络中输入层后的所述预处理层对所述目标训练图像进行预处理,得到预处理数据;利用所述像素重组层对所述预处理数据进行像素重组处理,得到重组数据;将所述重组数据输入所述像素重组层后的后续网络层,得到所述输出数据;所述后续网络层包括多个网络层组,每个网络层组由一个卷积层和一个激活函数层组成。
- 根据权利要求5或6所述的图像处理方法,其中,所述后续网络层中最后一个所述网络层组的所述激活函数层为输出层,所述输出层的输出区间为预设区间。
- 一种图像处理装置,其中,包括:获取模块,用于获取待处理图像;第一压缩模块,用于利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵;所述行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的所述目标非零元素,所述目标非零元素在各个所述行向量中的位置不同;第二压缩模块,用于利用所述行阶梯观测矩阵的转置矩阵对所述第一矩阵进行右乘处理,得到压缩数据。
- 根据权利要求8所述的图像处理装置,其中,所述第一压缩模块,包括:拆分单元,用于根据行阶梯观测矩阵的阶数对待处理图像进行拆分,得到若干个待处理子图像;第一压缩单元,用于利用行阶梯观测矩阵分别对各个待处理子图像进行左乘处理,得到若干个第一矩阵;相应的,所述第二压缩模块,包括:第二压缩单元,用于利用转置矩阵分别对各个第一矩阵进行右乘处理,得到若干个子压缩数据;拼接单元,用于对子压缩数据进行拼接,得到压缩数据。
- 根据权利要求8所述的图像处理装置,其中,还包括:生成模块,用于基于初始网络生成重构网络;所述初始网络为卷积神经网络,所述初始网络利用上采样处理层替代全连接层;输入模块,用于将压缩数据输入重构网络,得到重构图像;输出模块,用于输出重构图像。
- 根据权利要求10所述的图像处理装置,其中,所述生成模块包括:目标训练图像确定模块,用于获取训练集,并从训练集中获取目标训练图像;损失值计算模块,用于将目标训练图像输入初始网络,得到输出结果,并利用输出结果基于第一损失函数计算损失值;函数替换模块,用于若损失值的下降幅度低于预设阈值,则将第一损失函数替换为第二损失函数;参数调节模块,用于根据损失值调节初始网络的网络参数,并更新目标训练图像,直至得到重构网络。
- 一种电子设备,其中,包括存储器和处理器,其中:所述存储器,用于保存计算机程序;所述处理器,用于执行所述计算机程序,以实现如下的图像处理方法:获取待处理图像;利用行阶梯观测矩阵对所述待处理图像进行左乘处理,得到第一矩阵;所述行阶梯观测矩阵由目标非零元素和零元素组成,各个行向量具有两个相邻的所述目标非零元素,所述目标非零元素在各个所述行向量中的位置不同;利用所述行阶梯观测矩阵的转置矩阵对所述第一矩阵进行右乘处理,得到压缩数据。
- 一种计算机可读存储介质,其中,用于保存计算机程序,其中,所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的图像处理方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110161169.5 | 2021-02-05 | ||
CN202110161169.5A CN112802139A (zh) | 2021-02-05 | 2021-02-05 | 一种图像处理方法、装置、电子设备及可读存储介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022166298A1 true WO2022166298A1 (zh) | 2022-08-11 |
Family
ID=75814472
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/130805 WO2022166298A1 (zh) | 2021-02-05 | 2021-11-16 | 一种图像处理方法、装置、电子设备及可读存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112802139A (zh) |
WO (1) | WO2022166298A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115330901A (zh) * | 2022-10-17 | 2022-11-11 | 广东工业大学 | 一种基于压缩感知网络的图像重构方法和装置 |
CN116342984A (zh) * | 2023-05-31 | 2023-06-27 | 之江实验室 | 一种模型训练的方法以及图像处理的方法及装置 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112802139A (zh) * | 2021-02-05 | 2021-05-14 | 歌尔股份有限公司 | 一种图像处理方法、装置、电子设备及可读存储介质 |
CN113538308A (zh) * | 2021-06-29 | 2021-10-22 | 上海联影医疗科技股份有限公司 | 图像数据处理方法、装置、计算机设备和存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103778919A (zh) * | 2014-01-21 | 2014-05-07 | 南京邮电大学 | 基于压缩感知和稀疏表示的语音编码方法 |
US9904740B1 (en) * | 2013-08-30 | 2018-02-27 | Hrl Laboratories, Llc | Network of networks reconstruction employing compressed sensing |
CN108171762A (zh) * | 2017-12-27 | 2018-06-15 | 河海大学常州校区 | 一种深度学习的压缩感知同类图像快速重构***与方法 |
CN111106836A (zh) * | 2019-10-30 | 2020-05-05 | 北京歌尔泰克科技有限公司 | 图像重构方法和装置 |
CN112802139A (zh) * | 2021-02-05 | 2021-05-14 | 歌尔股份有限公司 | 一种图像处理方法、装置、电子设备及可读存储介质 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107991636B (zh) * | 2017-11-24 | 2020-05-12 | 哈尔滨工业大学 | 一种基于适应性结构低秩矩阵的快速磁共振图像重建方法 |
CN109544560B (zh) * | 2018-10-31 | 2021-04-27 | 上海商汤智能科技有限公司 | 图像处理方法及装置、电子设备和存储介质 |
CN110110715A (zh) * | 2019-04-30 | 2019-08-09 | 北京金山云网络技术有限公司 | 文本检测模型训练方法、文本区域、内容确定方法和装置 |
CN111953888B (zh) * | 2019-05-16 | 2021-12-24 | 武汉Tcl集团工业研究院有限公司 | 暗光成像方法、装置、计算机可读存储介质及终端设备 |
CN110943944A (zh) * | 2019-11-07 | 2020-03-31 | 浙江大学 | 一种基于深度学习的ofdm***的信道估计方法 |
-
2021
- 2021-02-05 CN CN202110161169.5A patent/CN112802139A/zh active Pending
- 2021-11-16 WO PCT/CN2021/130805 patent/WO2022166298A1/zh active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9904740B1 (en) * | 2013-08-30 | 2018-02-27 | Hrl Laboratories, Llc | Network of networks reconstruction employing compressed sensing |
CN103778919A (zh) * | 2014-01-21 | 2014-05-07 | 南京邮电大学 | 基于压缩感知和稀疏表示的语音编码方法 |
CN108171762A (zh) * | 2017-12-27 | 2018-06-15 | 河海大学常州校区 | 一种深度学习的压缩感知同类图像快速重构***与方法 |
CN111106836A (zh) * | 2019-10-30 | 2020-05-05 | 北京歌尔泰克科技有限公司 | 图像重构方法和装置 |
CN112802139A (zh) * | 2021-02-05 | 2021-05-14 | 歌尔股份有限公司 | 一种图像处理方法、装置、电子设备及可读存储介质 |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115330901A (zh) * | 2022-10-17 | 2022-11-11 | 广东工业大学 | 一种基于压缩感知网络的图像重构方法和装置 |
CN115330901B (zh) * | 2022-10-17 | 2023-01-17 | 广东工业大学 | 一种基于压缩感知网络的图像重构方法和装置 |
CN116342984A (zh) * | 2023-05-31 | 2023-06-27 | 之江实验室 | 一种模型训练的方法以及图像处理的方法及装置 |
CN116342984B (zh) * | 2023-05-31 | 2023-08-08 | 之江实验室 | 一种模型训练的方法以及图像处理的方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN112802139A (zh) | 2021-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022166298A1 (zh) | 一种图像处理方法、装置、电子设备及可读存储介质 | |
CN108022212B (zh) | 高分辨率图片生成方法、生成装置及存储介质 | |
US20210248715A1 (en) | Method and system for end-to-end image processing | |
WO2019192316A1 (zh) | 图像相关处理方法及装置、设备、存储介质 | |
WO2022110638A1 (zh) | 人像修复方法、装置、电子设备、存储介质和程序产品 | |
WO2019184823A1 (zh) | 基于卷积神经网络模型的图像处理方法和装置 | |
CN109544448B (zh) | 一种拉普拉斯金字塔结构的团网络超分辨率图像重建方法 | |
EP4030379A1 (en) | Image processing method, smart device, and computer-readable storage medium | |
CN112991203B (zh) | 图像处理方法、装置、电子设备及存储介质 | |
CN110533594B (zh) | 模型训练方法、图像重建方法、存储介质及相关设备 | |
WO2023284401A1 (zh) | 图像美颜处理方法、装置、存储介质与电子设备 | |
CN111510739B (zh) | 一种视频传输方法及装置 | |
WO2022206202A1 (zh) | 图像美颜处理方法、装置、存储介质与电子设备 | |
KR102327972B1 (ko) | 프로젝션 이미지 구성 방법 및 디바이스 | |
WO2022247568A1 (zh) | 一种图像恢复方法、装置和设备 | |
WO2020248294A1 (zh) | 一种逆色调映射方法、装置及电子设备 | |
CN107547773B (zh) | 一种图像处理方法、装置及设备 | |
WO2021258530A1 (zh) | 图像分辨率处理方法、装置、设备及可读存储介质 | |
CN107220934B (zh) | 图像重建方法及装置 | |
CN109102463B (zh) | 一种超分辨率图像重建方法及装置 | |
CN111861940A (zh) | 一种基于条件连续调节的图像调色增强方法 | |
TWI788172B (zh) | 影像處理裝置以及方法 | |
CN115941966A (zh) | 一种视频压缩方法及电子设备 | |
CN115564655A (zh) | 基于深度学习的视频超分辨率重建方法、***及介质 | |
CN111553961B (zh) | 线稿对应色图的获取方法和装置、存储介质和电子装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21924294 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21924294 Country of ref document: EP Kind code of ref document: A1 |