CN113744138A

CN113744138A - Image processing method, image processing apparatus, and storage medium

Info

Publication number: CN113744138A
Application number: CN202010476609.1A
Authority: CN
Inventors: 夏文韬
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2020-05-29
Filing date: 2020-05-29
Publication date: 2021-12-03

Abstract

The present disclosure relates to an image processing method, an image processing apparatus, and a storage medium. The image processing method comprises the following steps: acquiring a single-channel gray image; interpolating a green channel pixel of the single-channel gray image through an interpolation algorithm to obtain a green channel horizontal direction pixel interpolation value and a green channel vertical direction pixel interpolation value of the single-channel gray image; carrying out adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by utilizing a pre-trained neural network model to obtain a green channel image of the single-channel gray image; and taking the green channel image as a reference template, and completing the red channel pixels and the blue channel pixels of the single-channel gray image by utilizing the interpolation algorithm to obtain a color image. By the method and the device, the color image with better image details and less pseudo colors is output after the image is demosaiced.

Description

Image processing method, image processing apparatus, and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, and a storage medium.

Background

The demosaick technology, also called demosaick algorithm, completes missing pixels in three channels of red (R), green (G), blue (B), can expand a single-channel gray image from a single-channel gray image into a three-channel color image, and adaptively enhances the definition of the single-channel gray image and weakens image noise according to a certain strategy.

With the development of the camera shooting technology, the requirements of users on the images presented after the images are shot through the camera shooting device are higher and higher. Furthermore, how to demosaic a single-channel gray image obtained after an image is shot better, and output an image with better image details and less pseudo colors is a new problem facing demosaic at present.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides an image processing method, an image processing apparatus, and a storage medium.

According to a first aspect of embodiments of the present disclosure, there is provided an image processing method including: acquiring a single-channel gray image; interpolating a green channel pixel of the single-channel gray image by an interpolation algorithm to obtain a green channel horizontal direction pixel interpolation value and a green channel vertical direction pixel interpolation value of the single-channel gray image; carrying out self-adaptive interpolation fusion on the pixel interpolation in the horizontal direction of the green channel and the pixel interpolation in the vertical direction of the green channel by using a pre-trained neural network model to obtain a green channel image of a single-channel gray image; and (3) taking the green channel image as a reference template, and completing the red channel pixels and the blue channel pixels of the single-channel gray image by utilizing an interpolation algorithm to obtain a color image.

In one example, the image processing method further includes: determining an image training set, wherein the image training set comprises color training images; preprocessing the color training images in the image training set to obtain preprocessed color training images; determining the preprocessed color training image as a contrast map of an output image of the neural network model; performing mosaic processing on the comparison image to obtain a single-channel gray level training image corresponding to the comparison image; and training the neural network model based on the single-channel gray training image and the contrast map.

In one example, the training set of images includes: the method comprises the following steps of acquiring one or more of high-definition images, high-frequency texture images, images shot by a specified camera module and images shot by a single-lens reflex camera based on a public high-definition image set.

In one example, the contrast map is obtained by performing brightness and noise removal and performing edge enhancement processing.

In one example, the pre-trained neural network model is a U-shaped neural network model.

According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus including: an acquisition unit configured to acquire a single-channel grayscale image; the processing unit is configured to interpolate green channel pixels of the single-channel gray image through an interpolation algorithm to obtain green channel horizontal direction pixel interpolation and green channel vertical direction pixel interpolation of the single-channel gray image, perform self-adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by using a pre-trained neural network model to obtain a green channel image of the single-channel gray image, and complement red channel pixels and blue channel pixels of the single-channel gray image through the interpolation algorithm by using the green channel image as a reference template to obtain a color image.

In one example, the image processing apparatus further includes: a determination unit configured to determine an image training set, the image training set including color training images therein; the processing unit is further configured to: preprocessing the color training images in the image training set to obtain preprocessed color training images; determining the preprocessed color training image as a contrast map of an output image of the neural network model; performing mosaic processing on the comparison image to obtain a single-channel gray level training image corresponding to the comparison image; and the training unit is configured to train the neural network model based on the single-channel gray scale training image and the contrast map.

According to a third aspect of the present disclosure, there is provided an image processing apparatus including: a memory configured to store instructions. And a processor configured to invoke instructions to perform the image processing method of the foregoing first aspect or any example of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a processor, perform the image processing method of the foregoing first aspect or any one of the examples of the first aspect.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: before the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image are spliced, the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image are corrected and fused by utilizing the self-adaptive interpolation characteristic of a pre-trained neural network model, the green channel image with high accuracy can be obtained, and then the color image with accurate color is output by utilizing an interpolation algorithm according to the green channel image with high accuracy. By the method and the device, the color image with better image details and less pseudo colors can be output after the demosaicing of the image is realized.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow diagram illustrating an image processing method according to an exemplary embodiment.

Fig. 2 is a schematic diagram of a structure of a pnet network employing the present disclosure, according to an example embodiment.

FIG. 3 is a flowchart illustrating a method for training a Unet model according to an example embodiment.

Fig. 4 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

FIG. 5 is a block diagram illustrating an apparatus in accordance with an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The technical solution of the exemplary embodiment of the present disclosure may be applied to an application scene in which shooting is performed using a terminal including a shooting function. In the exemplary embodiments described below, a terminal is sometimes also referred to as an intelligent terminal device, where the terminal may be a Mobile terminal, and may also be referred to as a User Equipment (UE), a Mobile Station (MS), and the like. A terminal is a device that provides voice and/or data connection to a user, or a chip disposed in the device, such as a handheld device, a vehicle-mounted device, etc. having a wireless connection function. Examples of terminals may include, for example: the Mobile terminal comprises a Mobile phone, a tablet computer, a notebook computer, a palm computer, Mobile Internet Devices (MID), a wearable device, a Virtual Reality (VR) device, an Augmented Reality (AR) device, a wireless terminal in industrial control, a wireless terminal in unmanned driving, a wireless terminal in remote operation, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home and the like.

Currently, the mainstream demosaick algorithm utilizes an interpolation algorithm to implement demosaick. The core idea is to take the interpolation result of the G channel as a reference, and predict the difference between the R, B channel and the G channel to complement the R, B channel. However, with the development of the camera technology, how to better demosaic, output images with better details and less false colors is a new problem facing demosaic at present.

Further, in order to further improve the effect achieved by demosaick using an interpolation algorithm, the hottest deep convolutional neural network in Artificial Intelligence (AI) is beginning to be used in demosaick. However, the convolutional neural network can improve the demosaicing performance, and meanwhile, the convolutional neural network has a complex structure, too many calling parameters and a low operation speed. This is the biggest problem of AI-demosaick today.

Based on this, the present disclosure provides an image processing method that can demosaic an image in combination with an interpolation algorithm and a convolutional neural network. On the premise of ensuring that the demosaicing processing speed of the image is high and meeting the commercial landing requirement, the demosaicing processing performance of the image is improved.

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment, the image processing method including the following steps, as shown in fig. 1.

In step S11, a single-channel grayscale image is acquired.

In the present disclosure, the single-channel grayscale image may be a single-channel grayscale image captured by an image sensor in the camera device when captured with the camera device. The single-channel gray scale image may be, for example, a single-channel gray scale image arranged in a bayer array.

The camera device may be a digital camera, a single lens reflex camera, a camera device included in a smart phone, or the like.

In step S12, the green channel pixels of the single-channel grayscale image are interpolated by an interpolation algorithm to obtain a green channel horizontal direction pixel interpolation and a green channel vertical direction pixel interpolation of the single-channel grayscale image.

In the present disclosure, the interpolation algorithm may be, for example, a bilinear interpolation algorithm, a bicubic interpolation algorithm, a Residual Interpolation (RI) interpolation algorithm, or the like. The Residual-like Interpolation algorithm may be, for example, a minimum-Laplacian Residual Interpolation (MLRI) or an Adaptive Residual Interpolation (ARI).

In the present disclosure, for example, the green channel pixels of a single-channel grayscale image may be interpolated as follows:

the single-channel grayscale image is split into an R-channel pixel image, a G-channel pixel image, and a B-channel pixel image. And performing Green Interpolation Correction (GIC) on the G channel pixel image of the single-channel gray level image, namely correcting the error of the G channel pixel image between two Green and red color components and the error between two Green and blue color components, and then interpolating the Green channel pixel subjected to GIC Correction by utilizing an Interpolation algorithm to obtain Green channel horizontal direction pixel Interpolation and Green channel vertical direction pixel Interpolation of the single-channel gray level image.

In step S13, a pre-trained neural network model is used to perform adaptive interpolation fusion on the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel grayscale image, so as to obtain a green channel image of the single-channel grayscale image.

At present, when an interpolation algorithm is used for demosaicing an image, the obtained G-channel horizontal direction pixel interpolation and the vertical direction pixel interpolation can be directly spliced to obtain a green channel image.

Because the G channel pixel interpolation is carried out through the interpolation algorithm, the obtained G channel horizontal direction pixel interpolation and the obtained G channel vertical direction pixel interpolation have errors, and the obtained G channel horizontal direction pixel interpolation and the obtained G channel vertical direction pixel interpolation are directly spliced, so that the obtained green channel image is inaccurate in color. And then, when the green channel image with inaccurate color is used as a reference template and the red channel pixel and the blue channel pixel of the single-channel gray image are compensated by utilizing an interpolation algorithm, the obtained red channel image color and the blue channel image color are also inaccurate.

Therefore, the method can correct the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image by utilizing the self-adaptive interpolation characteristic of the pre-trained neural network model before the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image are spliced, and then fuse the corrected interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel to obtain the green channel image with high accuracy.

The pre-trained Neural Network model may be, for example, a U-shaped Neural Network model (uet) or a demosaicing Convolutional Neural Network (DmCNN).

In the present disclosure, taking a pre-trained neural network model as an example of the Unet, the green channel image of the single-channel grayscale image is obtained by performing adaptive interpolation fusion on the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel grayscale image by using the pre-trained neural network model, and is described below.

Because the Unet model can carry out self-adaptive correction on the horizontal pixel channel and the vertical pixel channel of the image after interpolation, after the errors of the pixel interpolation in the horizontal direction and the pixel interpolation in the vertical direction of the G channel are corrected by using the Unet model, the pixel interpolation in the horizontal direction and the pixel interpolation in the vertical direction of the G channel after the errors are corrected are fused to obtain the accurate green channel image of the single-channel gray image.

The single-channel gray image demosaicing is carried out by utilizing an interpolation algorithm and a Unet model, for example, the demosaicing technology can be applied to a digital image demosaicing technology developed aiming at an ISP chip in a terminal, and the single-channel gray image in a Bayer array mode is restored into a complete red, blue and green three-channel color image.

Fig. 2 is a schematic diagram of a structure of a pnet network using the present disclosure. In fig. 2, since the Unet model only needs to fuse the horizontal direction pixel interpolation and the vertical direction pixel interpolation of the green channel, the number of convolution box layers of the Unet model only needs to be 6. After the image is input into the Unet model, the image is output through an encoder (encoder) and a decoder (decoder). The encoder (decoder) part comprises a triple layer convolution box (conv Block), and then the image is passed through the decoder part by the encoder and correspondingly also comprises a triple layer convolution box (conv Block). There may be a shortcut connection (short cut) between the encoder and the decoder to increase the learning performance of the Unet model for the image features.

In addition, in order to enhance the invariance of image scale, up-down sampling is added between every two convolution boxes of the Unet model, namely when the green channel horizontal direction pixel interpolation and the vertical direction pixel interpolation are fused through the Unet model disclosed by the invention, an amplified single-channel gray image obtained by amplifying a single-channel gray image and a thumbnail single-channel gray image obtained by reducing the single-channel gray image are added between every two convolution boxes, so that when the green channel horizontal direction pixel interpolation and the vertical direction pixel interpolation are fused, the multi-scale fusion of images can be realized.

In step S14, the green channel image is used as a reference template, and the red channel pixel and the blue channel pixel of the single-channel grayscale image are complemented by using an interpolation algorithm, so as to obtain a color image.

According to the method, after the accurate green channel image is obtained by fusing the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray image through the pre-trained neural network model, the red channel pixel and the blue channel pixel of the single-channel gray image are completed by utilizing a residual interpolation algorithm according to the accurate green channel image, the accurate red channel image and the accurate blue channel image are obtained, and then the color image is output.

For example, complementing red channel pixels of a single-channel grayscale image with a residual interpolation algorithm may be implemented as follows:

and taking the green channel image fused by the Unet model as a reference image, performing guided upsampling to obtain an initialized red channel image, performing operation on the initialized red channel image and red channel pixels of the single-channel gray image to obtain a minimum Laplace residual error, and finally adding an upsampled residual error result back to the red channel pixels of the single-channel gray image to obtain the red channel image of the single-channel gray image.

Similarly, the green channel image fused by the Unet model is used as a reference image, and the reference image and the blue channel pixel are complemented to obtain a blue channel image of the single-channel gray image. And combining the obtained green channel image of the single-channel gray level image, the obtained blue channel image of the single-channel gray level image and the obtained red channel image of the single-channel gray level image to obtain an output color image.

In the exemplary embodiment of the disclosure, before the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image are spliced, the interpolation of the pixels in the horizontal direction and the interpolation of the pixels in the vertical direction of the green channel of the single-channel gray-scale image are corrected and fused by using the adaptive interpolation characteristic of the pre-trained neural network model, so that a green channel image with high accuracy can be obtained, and then a color image with accurate color is output by using a residual interpolation algorithm according to the green channel image with high accuracy. By the method and the device, the color image with better image details and less pseudo colors can be output after the demosaicing of the image is realized.

In the present disclosure, a neural network model may be trained prior to demosaicing a single-channel grayscale image with an interpolation algorithm and a pre-trained neural network model to obtain a color image.

The present disclosure will be described below with reference to a neural network model as an example of a Unet model, which is a neural network model for training demosaicing.

Fig. 3 is a flowchart illustrating a method for training a pnet model according to an exemplary embodiment, and as shown in fig. 3, the method for training the pnet model includes the following steps.

In step S21, an image training set is determined.

Because the training set plays an important role in training the neural network model, in order to enable the demosaicing effect of the trained Unet model to be better, the image training set can be acquired in a targeted manner according to the camera module to which demosaicing needs to be applied. For example, according to the parameters installed in the terminal a camera module, the image training set suitable for the terminal a camera module can be determined with pertinence. And then when training, more have directive property for the performance of training the Unet model will be better.

Wherein the image training set comprises color training images. The colored training images may be based on one or more of high-definition images acquired by a common high-definition image set, high-frequency texture images, images captured by a designated camera module, and images captured by a single-lens reflex camera.

The high-definition images acquired through the public high-definition image set can be used for initializing parameters of the Unet model, and the parameters of the Unet model are fixed in a certain range. The picture with high-frequency texture from the design can be used for improving the texture recovery capability of the Unet model on the image. Through appointing the module of making a video recording that demosaicing needs to be applied to promptly, the image of shooing can be used to improve the robustness of Unet model. The image shot by the single-lens reflex camera can be used for improving the image quality enhancement capability of the Unet model, such as improving the noise reduction capability of the image and improving the texture definition of the image.

Therefore, when the Unet model is trained through the image training set, the image training set is specially designed for being applied to the appointed camera module, the Unet model can be trained according to the targeted image training set, the trained Unet model can have obvious pertinence and directivity, and the images can be processed more quickly when the trained Unet model is deployed to a digital image processing platform applying the appointed camera module. Moreover, based on demosaicing of images in the appointed camera module, the obtained color image is more real in color and weaker in noise intensity, so that the color image output through the Unet model has higher definition, and better color rendition can be output to the image in extremely dark and bright extreme illumination environments.

In step S22, a contrast map of the single-channel grayscale training image and the Unet model output image is obtained from the image training set.

In one embodiment, the present disclosure may preprocess a color training image in an image training set to obtain a preprocessed color training image. The preprocessing may include, for example, at least one of degamma and white balance of the color training image.

In order to improve the training speed of the Unet model, the present disclosure may clip the preprocessed color training images into small images, for example, the preprocessed color training images may be clipped into small images with pixel size of 30 × 30, so as to obtain a contrast image of the output image of the Unet model. Moreover, if the resolution of the preprocessed color training image exceeds the maximum value required by the training designated camera module, for example, for an image shot by a single-lens reflex camera or a high-definition image acquired based on a public high-definition image set, the image can be downsampled to a proper size and then cut. For example, down-sampling the image into an image with a pixel dot size of 3000 x 4000.

According to the method, after a contrast diagram of an output image of the Unet model is obtained, mosaic processing is carried out on the contrast diagram, and a single-channel gray level training image which is input into the U-shaped neural network and corresponds to the contrast diagram is obtained.

In one embodiment, in order to enable the trained Unet model to enhance the image and reduce noise while demosaicing, the present disclosure may further perform brightness and noise removal on the contrast map and perform edge enhancement processing to obtain a processed contrast map. And training the Unet model based on the processed contrast image and the single-channel gray level training image.

In step S23, a U-shaped neural network model is trained based on the single-channel grayscale training image and the contrast map.

And inputting the single-channel gray training image and the comparison image into the Unet model, and predicting the color image of the single-channel gray training image through the Unet model to obtain a color prediction image corresponding to the single-channel gray training image. And calculating the error between the color predicted image and the contrast image according to the loss function, and adjusting the parameters of the Unet model through the calculated error until the error calculated through the loss function is lower than a preset threshold value to obtain the trained U-shaped neural network model.

In the exemplary embodiment of the disclosure, by acquiring the image training set for the designated camera module, when the Unet model is trained, since the image training set is specially designed for the application to the designated camera module, the Unet model is trained according to the targeted image training set, so that the trained Unet model has more obvious pertinence and directivity, and when the trained Unet model is deployed to the digital image processing platform to which the designated camera module is applied, the image can be processed more quickly. Moreover, based on demosaicing of images in the appointed camera module, the obtained color image is more real in color and weaker in noise intensity, so that the color image output through the Unet model has higher definition, and better color rendition can be output to the image in extremely dark and bright extreme illumination environments.

Based on the same inventive concept, the present disclosure also provides an image processing apparatus.

It is understood that, in order to implement the above functions, the application control device provided in the embodiments of the present disclosure includes a hardware structure and/or a software module corresponding to each function. The disclosed embodiments can be implemented in hardware or a combination of hardware and computer software, in combination with the exemplary elements and algorithm steps disclosed in the disclosed embodiments. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Fig. 4 is a block diagram 400 illustrating an image processing apparatus according to an exemplary embodiment. Referring to fig. 4, the image processing apparatus includes an acquisition unit 401 and a processing unit 402.

Wherein, the acquiring unit 401 is configured to acquire a single-channel grayscale image; the processing unit 402 is configured to interpolate green channel pixels of the single-channel grayscale image by an interpolation algorithm to obtain green channel horizontal direction pixel interpolation and green channel vertical direction pixel interpolation of the single-channel grayscale image, perform adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by using a pre-trained neural network model to obtain a green channel image of the single-channel grayscale image, and perform complementation on red channel pixels and blue channel pixels of the single-channel grayscale image by using the green channel image as a reference template to obtain a color image.

In one example, the image processing apparatus further includes: a determining unit 403 configured to determine an image training set, the image training set including color training images therein; the processing unit 402 is further configured to: preprocessing the color training images in the image training set to obtain preprocessed color training images; determining the preprocessed color training image as a contrast map of an output image of the neural network model; performing mosaic processing on the comparison image to obtain a single-channel gray level training image corresponding to the comparison image; a training unit 404 configured to train the neural network model based on the single-channel grayscale training image and the contrast map.

In an example, the processing unit 402 is further configured to: the contrast image is a processed contrast image obtained after brightness and noise removal and edge enhancement processing are performed.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 5 is a block diagram illustrating an apparatus 500 for image processing according to an exemplary embodiment. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 5, the apparatus 500 may include one or more of the following components: processing component 502, memory 504, power component 505, multimedia component 508, audio component 510, input/output (I/O) interface 512, sensor component 514, and communication component 516.

The processing component 502 generally controls overall operation of the device 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 502 may include one or more processors 520 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 502 can include one or more modules that facilitate interaction between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

The memory 504 is configured to store various types of data to support operations at the apparatus 500. Examples of such data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 504 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 505 provides power to the various components of the device 500. The power components 505 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power supplies for the apparatus 500.

The multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 500 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 510 is configured to output and/or input audio signals. For example, audio component 510 includes a Microphone (MIC) configured to receive external audio signals when apparatus 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 504 or transmitted via the communication component 516. In some embodiments, audio component 510 further includes a speaker for outputting audio signals.

The I/O interface 512 provides an interface between the processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the device 500. For example, the sensor assembly 514 may detect an open/closed state of the apparatus 500, the relative positioning of the components, such as a display and keypad of the apparatus 500, the sensor assembly 514 may also detect a change in the position of the apparatus 500 or a component of the apparatus 500, the presence or absence of user contact with the apparatus 500, orientation or acceleration/deceleration of the apparatus 500, and a change in the temperature of the apparatus 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 516 is configured to facilitate communication between the apparatus 500 and other devices in a wired or wireless manner. The apparatus 500 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 504 comprising instructions, executable by the processor 520 of the apparatus 500 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

It is further understood that the use of "a plurality" in this disclosure means two or more, as other terms are analogous. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. The singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It will be further understood that the terms "first," "second," and the like are used to describe various information and that such information should not be limited by these terms. These terms are only used to distinguish one type of information from another and do not denote a particular order or importance. Indeed, the terms "first," "second," and the like are fully interchangeable. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure.

It is further to be understood that while operations are depicted in the drawings in a particular order, this is not to be understood as requiring that such operations be performed in the particular order shown or in serial order, or that all illustrated operations be performed, to achieve desirable results. In certain environments, multitasking and parallel processing may be advantageous.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that the method comprises:

acquiring a single-channel gray image;

interpolating a green channel pixel of the single-channel gray image through an interpolation algorithm to obtain a green channel horizontal direction pixel interpolation value and a green channel vertical direction pixel interpolation value of the single-channel gray image;

carrying out adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by utilizing a pre-trained neural network model to obtain a green channel image of the single-channel gray image;

and taking the green channel image as a reference template, and completing the red channel pixels and the blue channel pixels of the single-channel gray image by utilizing the interpolation algorithm to obtain a color image.

2. The image processing method according to claim 1, characterized in that the method further comprises:

determining an image training set, wherein the image training set comprises color training images;

preprocessing the color training images in the image training set to obtain preprocessed color training images;

determining the preprocessed color training image as a contrast map of the output image of the neural network model;

performing mosaic processing on the comparison image to obtain a single-channel gray level training image corresponding to the comparison image;

training the neural network model based on the single-channel grayscale training image and the contrast map.

3. The image processing method of claim 2, wherein the training set of images comprises:

the method comprises the following steps of acquiring one or more of high-definition images, high-frequency texture images, images shot by a specified camera module and images shot by a single-lens reflex camera based on a public high-definition image set.

4. The image processing method according to claim 2, wherein the contrast map is obtained by performing luminance and noise removal and performing edge enhancement processing.

5. The image processing method according to any one of claims 1 to 4, wherein the neural network model is a U-shaped neural network model.

6. An image processing apparatus, characterized in that the apparatus comprises:

an acquisition unit configured to acquire a single-channel grayscale image;

a processing unit configured to interpolate green channel pixels of the single-channel grayscale image by an interpolation algorithm to obtain green channel horizontal direction pixel interpolation and green channel vertical direction pixel interpolation of the single-channel grayscale image, an

Performing adaptive interpolation fusion on the green channel horizontal direction pixel interpolation and the green channel vertical direction pixel interpolation by using a pre-trained neural network model to obtain a green channel image of the single-channel gray image, and

7. The image processing apparatus according to claim 6, characterized in that the apparatus further comprises:

a determination unit configured to determine an image training set, the image training set including color training images;

the processing unit is further configured to:

a training unit configured to train the neural network model based on the single-channel grayscale training image and the contrast map.

8. The image processing apparatus according to claim 7, wherein the training set of images comprises:

9. The image processing apparatus according to claim 7, wherein the contrast map is obtained by performing luminance and noise removal and performing edge enhancement processing.

10. The image processing apparatus according to any one of claims 6 to 9, wherein the pre-trained neural network model is a U-shaped neural network model.

11. An image processing apparatus characterized by comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: performing the image processing method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a processor, perform the image processing method of any one of claims 1-5.