CN116843583A

CN116843583A - Image processing method, device, electronic equipment and storage medium

Info

Publication number: CN116843583A
Application number: CN202311116889.5A
Authority: CN
Inventors: 杨佼汪
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-09-01
Filing date: 2023-09-01
Publication date: 2023-10-03
Anticipated expiration: 2043-09-01
Also published as: CN116843583B

Abstract

The embodiment of the application provides an image processing method, an image processing device, electronic equipment and a storage medium, wherein the method comprises the following steps: inputting an image to be processed and an image modification type into a parameter prediction model, and performing parameter prediction by the parameter prediction model according to the image to be processed and the image modification type, and outputting image modification parameters corresponding to the image to be processed and the image modification type; inputting the image to be processed and the image modification parameters into an image processing model, and processing the image to be processed by the image processing model according to the image modification parameters, and outputting the processed image. The image processing method provided by the application can realize automatic modification of the image without manual participation.

Description

Image processing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method, an image processing device, an electronic device, and a storage medium.

Background

In the field of professional photography, image retouching is a time-consuming process, often requiring fine adjustments manually. However, with the development of technology, automated image modification is becoming a demand. However, the existing automatic image modification method basically adopts a single model algorithm, and when the image is processed, smooth processing is often performed blindly, so that the detail texture and the depth effect of the image are destroyed.

To obtain better image processing effect, the user may empirically adjust the image modification parameters in the model, especially the local image modification parameters (e.g. foreground, background, subject corresponding image modification parameters). However, the image modification method requires the user to participate in the adjustment of the image modification parameters, so that the degree of automation is insufficient, and the user experience is affected.

It should be noted that the information disclosed in the background section of the present application is only for enhancement of understanding of the general background of the present application and should not be taken as an admission or any form of suggestion that this information forms the prior art that is well known to a person skilled in the art.

Disclosure of Invention

In view of the above, the present application provides an image processing method, an apparatus, an electronic device, and a storage medium, so as to solve the problem in the prior art that users need to participate in adjustment of image modification parameters, resulting in insufficient automation degree and influence on user experience.

In a first aspect, an embodiment of the present application provides an image processing method, including:

inputting an image to be processed and an image modification type into a parameter prediction model, and performing parameter prediction by the parameter prediction model according to the image to be processed and the image modification type, and outputting image modification parameters corresponding to the image to be processed and the image modification type;

Inputting the image to be processed and the image modification parameters into an image processing model, and processing the image to be processed by the image processing model according to the image modification parameters, and outputting the processed image.

In one possible implementation, the parameter prediction model includes an image recognition model and a parameter generation model;

inputting the image to be processed and the image modification type into a parameter prediction model, performing parameter prediction by the parameter prediction model according to the image to be processed and the image modification type, and outputting image modification parameters corresponding to the image to be processed and the image modification type, wherein the parameter prediction model comprises the following steps:

inputting an image to be processed and an image modification type into an image recognition model, wherein the image recognition model performs image recognition according to the image to be processed and the image modification type, and outputs image descriptions corresponding to the image to be processed and the image modification type;

inputting the image description into the parameter generation model, and performing parameter generation by the parameter generation model according to the image description, and outputting image modification parameters corresponding to the image description.

In one possible implementation manner, the inputting the image to be processed and the image modification parameter into an image processing model, the image processing model processes the image to be processed according to the image modification parameter, and outputs a processed image, including:

Inputting the image to be processed and the image modification parameters into a target image processing model corresponding to the image modification parameters, processing the image to be processed by the target image processing model according to the image modification parameters, and outputting a processed image.

In one possible implementation manner, the inputting the image to be processed and the image modification parameter into a target image processing model corresponding to the image modification parameter, where the target image processing model processes the image to be processed according to the image modification parameter, and outputs a processed image, including:

respectively inputting n types of the image modification parameters into n target image processing models, wherein each target image processing model corresponds to one type of the image modification parameters, and n is more than or equal to 2;

and the n target image processing models sequentially process the images to be processed and output the processed images.

In one possible implementation manner, the n target image processing models sequentially process the image to be processed, output a processed image, and include:

determining an image processing sequence according to the n types of the image modification parameters;

And the n target image processing models sequentially process the images to be processed according to the image processing sequence, and output the processed images.

In one possible implementation manner, the image processing types corresponding to a first target image processing model and a second target image processing model in the n target image processing models are the same or different, wherein the first target image processing model and the second target image processing model are any two target image processing models in the n target image processing models.

In one possible implementation, the image processing types include: flawed, refined, detail processed, hue adjusted, brightness adjusted, contrast adjusted, saturation adjusted, and/or color balance adjusted.

In a second aspect, an embodiment of the present application provides an image processing apparatus including:

the parameter prediction unit is used for receiving an image to be processed and an image modification type, performing parameter prediction according to the image to be processed and the image modification type, and outputting image modification parameters corresponding to the image to be processed and the image modification type;

the image processing unit is used for receiving the image to be processed and the image modification parameters, processing the image to be processed according to the image modification parameters and outputting the processed image.

In a third aspect, an embodiment of the present application provides an electronic device, including:

a processor;

a memory;

the memory has stored therein a computer program which, when executed, causes the electronic device to perform the method of any of the first aspects.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium includes a stored program, where the program when executed controls a device in which the computer readable storage medium is located to perform the method of any one of the first aspects.

Compared with the prior art, the embodiment of the application generates the image modification parameters through the parameter prediction model, and then transmits the generated image modification parameters to the image processing model to guide the image processing model to carry out image modification, thereby improving the degree of automation and saving a large amount of time and cost for manual adjustment.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an electronic device according to an embodiment of the present application;

fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;

FIG. 3 is a schematic diagram illustrating the operation of a parameter prediction model according to an embodiment of the present application;

FIG. 4 is training data of a parameter prediction model according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating the operation of another parameter prediction model according to an embodiment of the present application;

FIG. 6 is training data of another parametric prediction model provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of image modification process according to an embodiment of the present application;

FIG. 8 is a training schematic diagram of a D & B refined image processing model according to an embodiment of the present application;

FIG. 9 is a block diagram of a flow structure of an image processing method according to an embodiment of the present application;

fig. 10 is a block diagram of an image processing apparatus according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For a better understanding of the technical solution of the present application, the following detailed description of the embodiments of the present application refers to the accompanying drawings.

It should be understood that the described embodiments are merely some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be understood that the term "and/or" as used herein is merely one way of describing an association of associated objects, meaning that there may be three relationships, e.g., a and/or b, which may represent: the first and second cases exist separately, and the first and second cases exist separately. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

Referring to fig. 1, a schematic diagram of an electronic device is provided in an embodiment of the present application. The terminal shown in fig. 1 is a mobile phone 100, that is, the image processing method provided in the embodiment of the present application may be applied to the mobile phone 100. It should be noted that, in addition to the mobile phone 100, the electronic device provided by the embodiments of the present application may further include other terminals, such as a tablet computer, a personal computer (personal computer, PC), a personal digital assistant (personal digital assistant, PDA), a smart watch, a netbook, a wearable electronic device, an augmented reality (augmented reality, AR) device, a Virtual Reality (VR) device, a vehicle-mounted terminal, a smart car, a robot, a smart glasses, a smart television, and the like; in some possible implementations, the electronic device may also be a server, cloud device, or the like.

In the field of professional photography, image retouching is a time-consuming process, often requiring fine adjustments manually. However, with the development of technology, automated image modification is becoming a demand. However, the existing automatic image modification method basically adopts a single model algorithm, and when the image is processed, smooth processing is often performed blindly, so that the detail texture and the depth effect of the image are destroyed. To obtain better image processing effect, the user may empirically adjust the image modification parameters in the model, especially the local image modification parameters (e.g. foreground, background, subject corresponding image modification parameters). However, the image modification method requires the user to participate in the adjustment of the image modification parameters, so that the degree of automation is insufficient, and the user experience is affected.

In view of the above problems, an embodiment of the present application provides an image processing method, where an image modification parameter is generated by a parameter prediction model, and the generated image modification parameter is transferred to an image processing model to guide the image processing model to perform image modification, so that the degree of automation is improved, and a great amount of time and cost for manual adjustment are saved. The following detailed description is directed to specific embodiments.

Referring to fig. 2, a flowchart of an image processing method according to an embodiment of the present application is shown. The method can be applied to the electronic device shown in fig. 1, and mainly comprises the following steps as shown in fig. 2.

Step S201: inputting the image to be processed and the image modification type into a parameter prediction model, and carrying out parameter prediction by the parameter prediction model according to the image to be processed and the image modification type, and outputting image modification parameters corresponding to the image to be processed and the image modification type.

The image to be processed in the embodiment of the application can be an original image which is shot by a user or acquired from other equipment and uploaded to electronic equipment such as a cloud end, a server and the like to be subjected to image modification, and the embodiment of the application is not particularly limited.

After uploading the image to be processed, the user can input an image modification type according to the need, and the image modification type can be, for example, a calling card style, a cartoon style, a sketch style and the like, or can be specific image modification requirements such as stain repair, composition adjustment, color adjustment, fine repair and the like. In one possible implementation manner, the user can directly input the required image modification type, or the image modification type can be set to be selectable, and the user selects the required image modification type from the drop-down list.

In specific implementation, after the parameter prediction model receives the image to be processed and the image modification type, the parameter prediction model outputs corresponding image modification parameters according to the image to be processed and the image modification type. Referring to fig. 3, a working schematic diagram of a parameter prediction model according to an embodiment of the present application is provided. As shown in fig. 3, after the image to be processed and the image modification type are input into the parameter prediction model, the parameter prediction model performs feature extraction on the image to be processed and the image modification type, performs image modification parameter prediction according to the extracted features, and finally outputs the image modification parameters corresponding to the image to be processed and the image modification type. It can be understood that the output image modification parameters of the embodiment of the application are personalized image modification parameters, that is, even if the image modification types are the same for different images, the obtained image modification parameters may be different, so that the parameter prediction model of the application has strong applicability.

It should be noted that, the method for extracting features of the image to be processed and the image modification type in the embodiment of the application may refer to related descriptions in the prior art, and for simplicity of description, the present application is not repeated.

In one possible implementation, the parameter prediction model may be a MiniGPT-4, BLIP-2, or other multi-modal neural network model. According to the embodiment of the application, the parameter prediction model required by the application can be obtained by adjusting the training data of the multi-modal neural network model and training the multi-modal neural network model through the adjusted training data. In particular, the image-text data pair shown in fig. 4 may be used as training data for parametric predictive model training. Specifically, first training data shown in fig. 4 is obtained, the first training data includes a first to-be-processed image, a first image modification type and a first image modification parameter corresponding to the first to-be-processed image and the first image modification type, the first to-be-processed image and the first image modification type are used as input of a to-be-trained parameter prediction model, the first image modification parameter is used as output of the to-be-trained parameter prediction model, and the to-be-trained parameter prediction model is trained to obtain the parameter prediction model.

In particular implementations, the image modification parameters are typically structured data, which can be directly used as input to the image processing model. The image processing model is described below. Exemplary image modification parameters include blemish, D & B refinement (partial brightness correction refinement), detail, hue, brightness, contrast, saturation, color balance, and the like.

Referring to fig. 5, a schematic working diagram of another parameter prediction model according to an embodiment of the present application is provided. In this embodiment, the parameter prediction model includes an image recognition model and a parameter generation model, and the corresponding generation process of the image modification parameter includes: inputting the image to be processed and the image modification type into an image recognition model, recognizing the image content of the image recognition model according to the image to be processed and the image modification type, outputting image description sentences corresponding to the image to be processed and the image modification type, inputting the image description sentences into a parameter generation model, and generating structural image modification parameters according to the image description sentences by the parameter generation model.

In particular implementations, an image description statement is a complete content description of an image to be processed, including characters, scenes, illumination, colors, and image modification parameters, among others. Taking fig. 5 as an example, the image description sentence may be: the main body in the image is a woman wearing white suspenders and black trousers, standing on a hillside, opening both hands, and looking at the scenery when the user is sunrise; the light source in the image is sunlight; the background includes green grass, blue sky, white cloud, hillside and big tree; from the view of the image, the image is shot in a backlight mode, and the light ratio in the image is 1:2; the image modification parameters are as follows: light sensation +50, hue +15, brightness-10, contrast +30, saturation +50, etc. Of course, according to different training data, the image recognition model can give more detailed image description sentences, such as expression of a person, whether an image scene is real, and the like, or respectively predict parameters of the image modification parameters according to a main body, a foreground and a background area of the image, and the like, and a user can adjust an output result according to own needs, so that the application does not have specific requirements.

In one possible implementation, the image recognition model may be a multimodal neural network model such as MiniGPT-4, BLIP-2, etc., the parameter generation model may be a generated Pre-training transducer model (GPT), or other neural network model that may perform keyword extraction on the image description statement to generate the structured image modification parameters.

It can be understood that the image recognition model and the parameter generation model in the embodiment of the application are both trained parameter prediction models. In the specific implementation, the image recognition model and the parameter generation model may be trained separately, or the image recognition model and the parameter generation model may be trained as a whole. When training the image recognition model and the parameter generation model as a whole, training data as shown in fig. 6 may be employed. Specifically, the second training data shown in fig. 6 is obtained, the second training data includes a second image to be processed, a second image modification type, and a second image modification parameter corresponding to the second image to be processed and the second image modification type, the second image to be processed and the second image modification type are used as input of an image recognition model to be trained, the second image modification parameter is used as output of a parameter generation model to be trained, the image recognition model to be trained and the parameter generation model to be trained are trained, and the image recognition model and the parameter generation model are obtained.

Compared with the prior parameter prediction model only comprising a multi-modal neural network model, the multi-modal neural network model directly outputs the image modification structured data, and the specificity is stronger, the embodiment of the application has the advantages that the parameter generation model is arranged behind the multi-modal neural network model, the parameter generation model is used for extracting the characteristics of the output of the multi-modal neural network, the image modification structured data is generated, the applicability is wider, and the migration is easier to realize other tasks.

In the specific implementation, in order to enhance the training effect, the image modification parameters output by the parameter prediction model are more specialized, and in the training process of the parameter prediction model, the image modification scheme description guided by a specialized photographer can be adopted as the image modification parameters in the data to be trained. Compared with the prior art, the embodiment of the application allows a photographer to define the decoration style, simultaneously allows the photographer to adjust the image description, and trains the parameter prediction model by taking the adjusted image description as the image decoration parameter to realize interactive image decoration.

Step S202: inputting the image to be processed and the image modification parameters into an image processing model, processing the image to be processed by the image processing model according to the image modification parameters, and outputting the processed image.

In the embodiment of the application, after the image processing model receives the image to be processed and the image modification parameters, the image to be processed is subjected to image modification according to the specific content of the image modification parameters, and the image after finishing the image modification is output.

In one possible implementation, there are a plurality of image processing models, each for performing a different image processing task, and therefore, the image modification parameters required for each image processing model are different. Specifically, after the image modification parameters are obtained, the image modification parameters are required to be input into a target image processing model corresponding to the image modification parameters, the target image processing model processes the image to be processed according to the image modification parameters, and the processed image is output.

In one possible implementation, the image modification parameters include n types, and the target image processing model includes n types accordingly. In the specific implementation, n types of image modification parameters are respectively input into n target image processing models, wherein each target image processing model corresponds to one type of image modification parameter, and n is more than or equal to 2; and after the n target image processing models obtain the image modification parameters of the corresponding types, sequentially processing the images to be processed, and outputting the processed images. Illustratively, inputting a kth preprocessed image and a kth type of image modification parameters into a kth target image processing model, wherein the kth target image processing model processes the kth preprocessed image according to the kth type of image modification parameters and outputs a (k+1) th preprocessed image; wherein the 1 st pretreatment image is an image to be treated, the n+1st pretreatment image is a treated image, and k is more than or equal to 1 and less than or equal to n.

In the embodiment of the application, since there are a plurality of target image processing models, and each target image processing model needs to be modified in sequence, before the target image processing model performs image processing, the processing sequence of the target image processing model needs to be determined. Specifically, determining an image processing sequence according to n image modification parameters; the n target image processing models sequentially process the images to be processed according to the image processing sequence, and output the processed images. In practical application, some image modification processes (image processing sequences) can be preset according to different types contained in the image modification parameters, and when the image modification parameters with the same type are encountered, the corresponding image modification processes are directly called. It can be understood that, for different images to be processed and image modification types, the obtained image modification parameters are different, so that the generated image modification processes are different, that is, the processing sequences of the obtained target image processing models are different.

Referring to fig. 7, a schematic diagram is generated for an image modification process according to an embodiment of the present application. As shown in fig. 7, after the parameter prediction model generates the image modification parameters, different image modification processes are generated according to the image modification parameters, where the image modification parameters include: removing flaws, finishing D & B, detail and tone, and correspondingly generating an image modification flow comprising: flaw removal, D & B finishing, detail processing, color matching, and correspondingly, the required target image processing model comprises the following steps: an imperfections-removed image processing model, a D & B finishing image processing model, a detail processing image processing model, and a toned image processing model; and inputting the image modification parameters into the corresponding target image processing models, and then performing image processing on the target image processing models according to the image modification flow. The image modification process may further include: removing flaws, carrying out D & B finishing 1, D & B finishing 2, carrying out detail processing and color mixing; d & B finishing, detail processing and color mixing; flaw removal, detail treatment and color matching; detail processing, D & B finishing 1, D & B finishing 2, color mixing and the like.

In one possible implementation manner, the image processing types corresponding to the first target image processing model and the second target image processing model in the n target image processing models are the same or different, wherein the first target image processing model and the second target image processing model are any two target image processing models in the n target image processing models. Specifically, when the image processing types of the first target image processing model and the second target image processing model are the same, the method can be realized by calling the corresponding target image processing model twice, and also can be realized by calling two identical target image processing models.

In one possible implementation, corresponding to the image modification parameter, the image processing type includes: flawed, D & B refined, detail processed, hue adjusted, brightness adjusted, contrast adjusted, saturation adjusted, and/or color balance adjusted, etc., the corresponding target image processing model includes: an inpainting image processing model, a D & B finishing image processing model, a detail processing image processing model, a hue adjustment image processing model, a brightness adjustment image processing model, a contrast adjustment image processing model, a saturation adjustment image processing model, and/or a color balance adjustment image processing model.

In one possible implementation manner, when the parameter prediction model outputs corresponding image modification parameters according to the main body, the foreground and the background respectively, the target image processing model of the present application may also perform the respective modification of the main body, the foreground and the background, and the target image processing model of the present application includes the main body image processing model, the foreground image processing model and the background image processing model. According to the embodiment of the application, the corresponding main body, foreground and background image processing models are arranged, so that the main body, foreground and background can be modified respectively in a self-adaptive manner, and the problem of blind global processing of an algorithm is avoided.

In one possible implementation, the target image processing model may be a deep learning model, which may be, by way of example, a deep neural network (deep neural network, DNN), a convolutional neural network model (convolutional neuron network, CNN), or a generative network model (Generative adversarial network, GAN).

It can be understood that the target image processing models in the embodiments of the present application are all image processing models that have been trained in advance. In practice, since the tasks of the respective target process models are different, it is necessary to train the respective target process models. Referring to fig. 8, a training schematic diagram of a D & B refined image processing model is provided in an embodiment of the present application. Specifically, third training data are obtained, the third training data comprise a third image to be processed, a third image modification parameter and a third output image, wherein the third output image is obtained by processing the third image to be processed according to the third image modification parameter, the third image to be processed and the third image modification parameter are used as input of a D & B refined image processing model to be trained, specifically, the third image modification parameter is input to a refined layer and a mixed mode layer of the D & B refined image processing model to be trained, the third output image is used as output of the D & B refined image processing model to be trained, and the D & B refined image processing model to be trained is trained, so that the D & B refined image processing model is obtained.

It should be noted that, in the embodiment of the present application, the training image modification parameters may be derived from a parameter prediction model, or may be derived from a professional photographer, which is not specifically required by the present application.

The training process of other target image processing models is similar to the training process of the D & B refined image processing model, and in specific implementation, model training is performed by referring to the training process of the D & B refined image processing model, so that other target image processing models can be obtained.

In order to facilitate understanding, the image processing method provided by the embodiment of the application is described in detail below in connection with a specific implementation manner.

Referring to fig. 9, a block diagram of a flow structure of an image processing method according to an embodiment of the present application is provided. As shown in fig. 9, the image modification type and the original image including the person are input into a multi-mode large model 901 for image modification parameter prediction, the multi-mode large model 901 identifies the age, sex, skin color, scene, illumination and the like of the person in the original image, and according to the image modification type, the image modification parameter prediction is performed on how to perform subsequent image modification, and the structured image modification parameters are output. Specifically, the image modification parameters include an flaw removal processing parameter, a D & B finishing parameter, a detail processing parameter, and a color matching parameter, so that the corresponding image modification process is flaw removal, D & B finishing, detail processing, color matching, and the required target image processing model includes: an imperfections image processing model 902, a D & B refined image processing model 903, a details processing image processing model 904, and a hueing image processing model 905. After determining the image modification flow and the target image processing model, inputting the image modification parameters into the corresponding target image processing model, for example, inputting the flaw removing processing parameters into the flaw removing image processing model 902, inputting the D & B finishing parameters into the D & B finishing image processing model 903, inputting the detail processing parameters into the detail processing image processing model 904, inputting the color matching parameters into the color matching image processing model 905, after each target image processing model receives the corresponding image modification parameters, performing image processing according to the image modification flow, namely, the flaw removing image processing model 902 performs flaw identification and restoration on an original image according to the flaw removing processing parameters, and outputs a flaw removed picture, the D & B finishing image processing model 903 performs skin detail shadow reconstruction on the flaw removed picture according to the D & B finishing parameters through a detail finishing and mixing mode, and outputs a finished picture, the detail processing image processing model 904 performs detail identification and mixing mode processing on the finished picture according to the detail processing parameters, and outputs a detail adjusted picture, and the color matching image processing model 905 performs style identification and curve and LUT (Look Up Table) adjustment on the detail adjusted picture according to the color matching parameters, and then outputs the processed image.

According to the embodiment of the application, the image personalized modification parameters are generated through the parameter prediction model, and then the generated parameters are transmitted to the image processing model to guide the image processing model to carry out image modification, so that the rapid modification of the image is realized, and a large amount of time and cost for manual adjustment are saved.

Corresponding to the above embodiment, the present application also provides an image processing apparatus.

Referring to fig. 10, a block diagram of an image processing apparatus according to an embodiment of the present application is provided. As shown in fig. 10, the image forming apparatus 1000 includes: a parameter prediction unit 1001, configured to receive an image to be processed and an image modification type, perform parameter prediction according to the image to be processed and the image modification type, and output an image modification parameter corresponding to the image to be processed and the image modification type; the image processing unit 1002 is configured to receive the image to be processed and the image modification parameter, process the image to be processed according to the image modification parameter, and output the processed image.

The specific content of the embodiments of the present application may be referred to the description of the embodiments of the method, and for brevity of description, the description is omitted herein.

Corresponding to the above embodiment, the present application also provides an electronic device comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the electronic device to perform part or all of the steps of the above method embodiments.

Referring to fig. 11, a schematic structural diagram of an electronic device according to an embodiment of the present application is provided. As shown in fig. 11, the electronic device 1100 may include a processor 1110, an external memory interface 1120, an internal memory 1121, a universal serial bus (universal serial bus, USB) interface 1130, a charge management module 1140, a power management module 1141, a battery 1142, an antenna 1, an antenna 2, a mobile communication module 1150, a wireless communication module 1160, an audio module 1170, a speaker 1170A, a receiver 1170B, a microphone 1170C, an earphone interface 1170D, a sensor module 1180, keys 1190, a motor 1191, an indicator 1192, a camera 1193, a display 1194, and a subscriber identification module (subscriber identification module, SIM) card interface 1195, etc. The sensor module 1180 may include a pressure sensor 1180A, a gyroscope sensor 1180B, a barometric sensor 1180C, a magnetic sensor 1180D, an acceleration sensor 1180E, a distance sensor 1180F, a proximity sensor 1180G, a fingerprint sensor 1180H, a temperature sensor 1180J, a touch sensor 1180K, an ambient light sensor 1180L, a bone conduction sensor 1180M, and the like.

It should be understood that the illustrated structure of the embodiment of the present application does not constitute a specific limitation on the electronic device 1100. In other embodiments of the application, the electronic device 1100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 1110 may include one or more processing units, such as: the processor 1110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in processor 1110 for storing instructions and data. In some embodiments, the memory in processor 1110 is a cache memory. The memory may hold instructions or data that has just been used or recycled by the processor 1110. If the processor 1110 needs to reuse the instruction or data, it may be called directly from the memory. Repeated accesses are avoided, reducing the latency of the processor 1110, and thus improving the efficiency of the system.

In some embodiments, processor 1110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.

The I2C interface is a bi-directional synchronous serial bus comprising a serial data line (SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 1110 may include multiple sets of I2C buses. The processor 1110 may be coupled to the touch sensor 1180K, charger, flash, camera 1193, etc., respectively, through different I2C bus interfaces.

The I2S interface may be used for audio communication. In some embodiments, processor 1110 may include multiple sets of I2S buses. The processor 1110 may be coupled to the audio module 1170 via an I2S bus to enable communication between the processor 1110 and the audio module 1170.

PCM interfaces may also be used for audio communication to sample, quantize and encode analog signals. In some embodiments, the audio module 1170 and the wireless communication module 1160 may be coupled via a PCM bus interface.

The UART interface is a universal serial data bus for asynchronous communications. The bus may be a bi-directional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 1110 with the wireless communication module 1160.

The MIPI interface may be used to connect processor 1110 with peripheral devices such as display 1194, camera 1193, and the like. The MIPI interfaces include camera serial interfaces (camera serial interface, CSI), display serial interfaces (display serial interface, DSI), and the like. In some embodiments, processor 1110 and camera 1193 communicate via a CSI interface to implement the shooting functionality of electronic device 1100. Processor 1110 and display 1194 communicate via a DSI interface to implement the display functionality of electronic device 1100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or as a data signal. In some embodiments, GPIO interfaces may be used to connect processor 1110 with cameras 1193, display 1194, wireless communication module 1160, audio module 1170, sensor module 1180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, an MIPI interface, etc.

The USB interface 1130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 1130 may be used to connect a charger to charge the electronic device 1100, or may be used to transfer data between the electronic device 1100 and a peripheral device.

It should be understood that the interfacing relationship between the modules illustrated in the embodiments of the present application is only illustrative, and is not meant to limit the structure of the electronic device 1100. In other embodiments of the present application, the electronic device 1100 may also employ different interfaces in the above embodiments, or a combination of interfaces.

The charge management module 1140 is used to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charge management module 1140 may receive a charging input of a wired charger through the USB interface 1130. In some wireless charging embodiments, the charge management module 1140 may receive a wireless charging input through a wireless charging coil of the electronic device 1100. The charging management module 1140 may also provide power to the terminal through the power management module 1141 while charging the battery 1142.

The power management module 1141 is configured to connect the battery 1142, the charge management module 1140 and the processor 1110. The power management module 1141 receives input from the battery 1142 and/or the charge management module 1140 and provides power to the processor 1110, the internal memory 1121, the display 1194, the camera 1193, the wireless communication module 1160, and the like. The power management module 1141 may also be configured to monitor battery capacity, battery cycle number, battery health (leakage, impedance), and other parameters.

The wireless communication functions of the electronic device 1100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 1150, the wireless communication module 1160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 1100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 1150 may provide a solution for wireless communication, including 2G/3G/4G/5G, as applied to the electronic device 1100. The mobile communication module 1150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 1150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, etc., on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 1150 may amplify the signal modulated by the modem processor and convert the signal into electromagnetic waves through the antenna 1 to radiate the electromagnetic waves. In some embodiments, at least some of the functional modules of the mobile communication module 1150 may be disposed in the processor 1110. In some embodiments, at least some of the functional modules of the mobile communication module 1150 may be provided in the same device as at least some of the modules of the processor 1110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 1170A, receiver 1170B, etc.), or displays images or video through display 1194.

The wireless communication module 1160 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field wireless communication technology (near field communication, NFC), infrared technology (IR), etc., for application on the electronic device 1100. The wireless communication module 1160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 1160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 1110. The wireless communication module 1160 may also receive a signal to be transmitted from the processor 1110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.

In some embodiments, antenna 1 and mobile communication module 1150 of electronic device 1100 are coupled, and antenna 2 and wireless communication module 1160 are coupled, such that electronic device 1100 may communicate with a network and other devices through wireless communication techniques. The wireless communication techniques may include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a beidou satellite navigation system (beidou navigation satellite system, BDS), a quasi zenith satellite system (quasi-zenith satellite system, QZSS) and/or a satellite based augmentation system (satellite based augmentation systems, SBAS).

The electronic device 1100 implements display functionality via a GPU, a display 1194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display 1194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 1110 may include one or more GPUs that execute program instructions to generate or change display information.

The display 1194 is used to display images, videos, or the like. The display 1194 includes a display panel. The display panel may employ a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED) or an active-matrix organic light-emitting diode (matrix organic light emitting diode), a flexible light-emitting diode (flex), a mini, a Micro led, a Micro-OLED, a quantum dot light-emitting diode (quantum dot light emitting diodes, QLED), or the like. In some embodiments, the electronic device 1100 may include 1 or N display screens 1194, where N is a positive integer greater than 1.

The electronic device 1100 may implement shooting functions through an ISP, a camera 1193, a video codec, a GPU, a display 1194, an application processor, and the like.

The ISP is used to process the data fed back by the camera 1193. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image.

The camera 1193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. In some embodiments, the electronic device 1100 may include 1 or N cameras 1193, where N is a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 1100 is selecting a bin, the digital signal processor is used to fourier transform the bin energy, or the like.

Video codecs are used to compress or decompress digital video. The electronic device 1100 may support one or more video codecs. Thus, the electronic device 1100 may play or record video in a variety of encoding formats, such as: dynamic picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural-network (NN) computing processor, and can rapidly process input information by referencing a biological neural network structure, for example, referencing a transmission mode between human brain neurons, and can also continuously perform self-learning. Applications such as intelligent awareness of the electronic device 1100 may be implemented by the NPU,

the external memory interface 1120 may be used to connect an external memory card, such as a Micro SD card, to enable expansion of the memory capabilities of the electronic device 1100. The external memory card communicates with the processor 1110 via an external memory interface 1120 to implement data storage functions. For example, files such as music, video, etc. are stored in an external memory card.

The internal memory 1121 may be used to store computer executable program code including instructions. The internal memory 1121 may include a storage program area and a storage data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data created during use of the electronic device 1100 (e.g., audio data, phonebook, etc.), and so on. In addition, the internal memory 1121 may include a high-speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), or the like. The processor 1110 performs various functional applications of the electronic device 1100 as well as data processing by executing instructions stored in the internal memory 1121, and/or instructions stored in a memory provided in the processor.

The electronic device 1100 may perform audio functions through an audio module 1170, a speaker 1170A, a receiver 1170B, a microphone 1170C, an ear speaker interface 1170D, an application processor, and the like. Such as music playing, recording, etc.

The audio module 1170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 1170 may also be used to encode and decode audio signals.

The speaker 1170A, also referred to as a "horn," is used to convert audio electrical signals into sound signals. The electronic device 1100 may listen to music, or to hands-free conversations, through the speaker 1170A.

A receiver 1170B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal. When the electronic device 1100 is answering a call or voice message, the voice can be received by placing the receiver 1170B close to the human ear.

Microphone 1170C, also referred to as a "microphone" or "microphone", is used to convert acoustic signals into electrical signals. When making a call or transmitting voice information, the user can sound near the microphone 1170C through his mouth, inputting a sound signal to the microphone 1170C. The electronic device 1100 may be provided with at least one microphone 1170C.

The earphone interface 1170D is used to connect a wired earphone. The earphone interface 1170D may be a USB interface 1130 or a 3.5mm open mobile terminal platform (open mobile terminal platform, OMTP) standard interface, a american cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

Pressure sensor 1180A is configured to sense a pressure signal and may convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 1180A may be disposed at display 1194. Pressure sensor 1180A is of many types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a capacitive pressure sensor comprising at least two parallel plates with conductive material. The capacitance between the electrodes changes when a force is applied to pressure sensor 1180A. The electronic device 1100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display 1194, the electronic device 1100 detects the intensity of the touch operation according to the pressure sensor 1180A. The electronic device 1100 may also calculate the location of the touch based on the detection signal of the pressure sensor 1180A.

The gyro sensor 1180B may be used to determine the motion pose of the electronic device 1100. In some embodiments, the angular velocity of the electronic device 1100 about three axes (i.e., x, y, and z axes) may be determined by the gyro sensor 1180B. The gyro sensor 1180B may be used for photographing anti-shake.

The air pressure sensor 1180C is used to measure air pressure. In some embodiments, the electronic device 1100 calculates altitude from barometric pressure values measured by the barometric pressure sensor 1180C, aiding in positioning and navigation.

The magnetic sensor 1180D includes a hall sensor. The electronic device 1100 may detect the opening and closing of the flip holster using the magnetic sensor 1180D.

The acceleration sensor 1180E may detect the magnitude of acceleration of the electronic device 1100 in various directions (typically three axes). The magnitude and direction of gravity may be detected when the electronic device 1100 is stationary. The method can also be used for identifying the gesture of the terminal, and is applied to the applications such as horizontal and vertical screen switching, pedometers and the like.

A distance sensor 1180F for measuring distance. The electronic device 1100 may measure distance by infrared or laser. In some embodiments, the electronic device 1100 may range using the distance sensor 1180F to achieve fast focus.

Proximity light sensor 1180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 1100 emits infrared light outward through the light emitting diode. The electronic device 1100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that an object is in the vicinity of the electronic device 1100.

Ambient light sensor 1180L is used to sense ambient light level. The electronic device 1100 may adaptively adjust the brightness of the display 1194 based on perceived ambient light.

The fingerprint sensor 1180H is used to collect a fingerprint. The electronic device 1100 may utilize the collected fingerprint characteristics to unlock the fingerprint, access the application lock, take a photograph of the fingerprint, answer an incoming call, etc.

The temperature sensor 1180J is used to detect temperature. In some embodiments, the electronic device 1100 performs a temperature processing strategy using the temperature detected by the temperature sensor 1180J.

The touch sensor 1180K, also referred to as a "touch device". The touch sensor 1180K may be disposed on the display 1194, and the touch sensor 1180K and the display 1194 form a touch screen, which is also referred to as a "touch screen". The touch sensor 1180K is used to detect a touch operation acting on or near it. The touch sensor may communicate the detected touch operation to the application processor to determine the touch event type. Visual output related to touch operations can be provided through the display 1194. In other embodiments, the touch sensor 1180K may also be disposed on a surface of the electronic device 1100 at a location different from the location of the display 1194.

Bone conduction sensor 1180M may acquire a vibration signal. In some embodiments, bone conduction sensor 1180M may acquire a vibration signal of a human vocal tract vibrating bone piece. The bone conduction sensor 1180M may also contact the pulse of the human body to receive the blood pressure pulsation signal.

The keys 1190 include a power on key, a volume key, etc. The keys 1190 may be mechanical keys. Or may be a touch key. The electronic device 1100 may receive key inputs, generate key signal inputs related to user settings and function controls of the electronic device 1100.

The motor 1191 may generate a vibratory alert. The motor 1191 may be used for incoming call vibration alerting or touch vibration feedback.

The indicator 1192 may be an indicator light that may be used to indicate a state of charge, a change in charge, an indication message, a missed call, a notification, etc.

The SIM card interface 1195 is for connecting a SIM card. The SIM card may be inserted into the SIM card interface 1195 or removed from the SIM card interface 1195 to enable contact and separation with the electronic device 1100. The electronic device 1100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 1195 may support Nano SIM cards, micro SIM cards, and the like. The same SIM card interface 1195 may be plugged into multiple cards at the same time. The types of the plurality of cards may be the same or different. The electronic device 1100 interacts with the network through the SIM card to perform functions such as talking and data communication. In some embodiments, the electronic device 1100 employs esims, namely: an embedded SIM card.

In a specific implementation, the present application further provides a computer storage medium, where the computer storage medium may store a program, where when the program runs, the program controls a device where the computer readable storage medium is located to execute some or all of the steps in the foregoing embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random-access memory (random access memory, RAM), or the like.

In a specific implementation, an embodiment of the present application further provides a computer program product, where the computer program product contains executable instructions, where the executable instructions when executed on a computer cause the computer to perform some or all of the steps in the above method embodiments.

In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relation of association objects, and indicates that there may be three kinds of relations, for example, a and/or B, and may indicate that a alone exists, a and B together, and B alone exists. Wherein A, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of the following" and the like means any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

Those of ordinary skill in the art will appreciate that the various elements and algorithm steps described in the embodiments disclosed herein can be implemented as a combination of electronic hardware, computer software, and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In several embodiments provided by the present invention, any of the functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely exemplary embodiments of the present invention, and any person skilled in the art may easily conceive of changes or substitutions within the technical scope of the present invention, which should be covered by the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image processing method, the method comprising:

2. The method of claim 1, wherein the parametric prediction model comprises an image recognition model and a parametric generation model;

3. The method of claim 1, wherein the inputting the image to be processed and the image modification parameters into an image processing model, the image processing model processing the image to be processed according to the image modification parameters, outputting a processed image, comprises:

4. The method according to claim 1, wherein the inputting the image to be processed and the image modification parameter into a target image processing model corresponding to the image modification parameter, the target image processing model processing the image to be processed according to the image modification parameter, outputting a processed image, includes:

5. The method according to claim 4, wherein the n target image processing models sequentially process the image to be processed, output a processed image, and include:

6. The method according to claim 4 or 5, wherein image processing types corresponding to a first target image processing model and a second target image processing model in the n target image processing models are the same or different, and wherein the first target image processing model and the second target image processing model are any two target image processing models in the n target image processing models.

7. The method of claim 6, wherein the image processing type comprises: flawed, refined, detail processed, hue adjusted, brightness adjusted, contrast adjusted, saturation adjusted, and/or color balance adjusted.

8. An image processing apparatus, comprising:

9. An electronic device, comprising:

a processor;

a memory;

the memory has stored therein a computer program which, when executed, causes the electronic device to perform the method of any of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored program, wherein the program, when run, controls a device in which the computer readable storage medium is located to perform the method of any one of claims 1-7.