CN111738958A

CN111738958A - Picture restoration method and device, electronic equipment and computer readable medium

Info

Publication number: CN111738958A
Application number: CN202010600862.3A
Authority: CN
Inventors: 刘鼎; 罗琳捷
Original assignee: ByteDance Inc
Current assignee: ByteDance Inc
Priority date: 2020-06-28
Filing date: 2020-06-28
Publication date: 2020-10-02
Anticipated expiration: 2040-06-28
Also published as: CN111738958B

Abstract

The disclosure provides a picture restoration method and device, electronic equipment and a computer readable medium, and relates to the technical field of image processing. The picture restoration method comprises the following steps: processing the picture to be repaired according to a preset mask picture corresponding to the picture to be repaired to obtain an initial picture; respectively downsampling the mask picture and the initial picture to respectively obtain a downsampling mask result picture and a downsampling initial result picture; respectively performing up-sampling on the lower sampling mask result graph and the lower sampling initial result graph to obtain an upper sampling mask result graph after the up-sampling of the lower sampling mask result graph and an upper sampling initial result graph after the up-sampling of the lower sampling initial result graph; and calculating based on the value of each pixel point of the upper-sampling initial result graph and the value of the pixel point corresponding to each pixel point of the upper-sampling initial result graph in the upper-sampling mask result graph to obtain the value of each pixel point of the restored picture. The technical scheme provided by the disclosure can greatly reduce the repair time of the picture.

Description

Picture restoration method and device, electronic equipment and computer readable medium

Technical Field

The embodiment of the disclosure relates to the technical field of image processing, in particular to a picture restoration method and device, an electronic device and a computer readable medium.

Background

With the development of social networks, the popularization of smart phones and the like, people can take pictures anytime and anywhere and store the pictures or share the pictures with friends. Due to the background of the photo taking, some scenes which are not desired by the user appear in the photo, for example, in various scenic spots, shadows of other tourists which are not desired to appear or other scenes which are not desired to be taken appear, the photo taking effect can be greatly influenced, the user wants to remove the undesired parts in the photo, and the effect of not missing the picture content is kept.

In the prior art, the pictures can be repaired, so that people or scenes which the user does not want to appear in the pictures are removed. In the prior art, when the picture is repaired, the repairing method is complex, the repairing time is too long, and the user experience is greatly influenced.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, a method for repairing a picture is provided, where the method includes:

processing the picture to be repaired according to a preset mask picture corresponding to the picture to be repaired to obtain an initial picture;

respectively carrying out downsampling on the mask picture and the initial picture to obtain a downsampled lower sampling mask result picture of the mask picture and a downsampled lower sampling initial result picture of the initial picture;

respectively performing up-sampling on the lower sampling mask result graph and the lower sampling initial result graph to obtain an upper sampling mask result graph after the up-sampling of the lower sampling mask result graph and an upper sampling initial result graph after the up-sampling of the lower sampling initial result graph;

and calculating based on the value of each pixel point of the upper-sampling initial result graph and the value of the pixel point corresponding to each pixel point of the upper-sampling initial result graph in the upper-sampling mask result graph to obtain the value of each pixel point of the repair picture.

In a second aspect, there is provided a picture restoration device, including:

the pre-processing module is used for processing the picture to be repaired according to a preset mask picture corresponding to the picture to be repaired to obtain an initial picture;

the downsampling processing module is used for respectively downsampling the mask picture and the initial picture to obtain a downsampled lower sampling mask result picture of the mask picture and a downsampled lower sampling initial result picture of the initial picture;

the upper sampling processing module is used for respectively carrying out upper sampling on the lower sampling mask result graph and the lower sampling initial result graph to obtain an upper sampling mask result graph after the upper sampling of the lower sampling mask result graph and an upper sampling initial result graph after the upper sampling of the lower sampling initial result graph;

and the restoration module is used for calculating based on the value of each pixel point of the upper-sampling initial result graph and the value of the pixel point corresponding to each pixel point of the upper-sampling initial result graph in the upper-sampling mask result graph so as to obtain the value of each pixel point of the restored picture.

In a third aspect, an electronic device is also provided, which includes:

one or more processors;

a memory;

one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the picture restoration method shown in the first aspect of the present disclosure is performed.

In a fourth aspect, a computer-readable medium is also provided, on which a computer program is stored, which when executed by a processor implements the picture restoration method shown in the first aspect of the present disclosure.

Compared with the prior art, the embodiment of the disclosure provides a picture restoration method, a device, an electronic device and a computer readable medium, wherein according to a preset mask picture corresponding to a picture to be restored, the picture to be restored is processed to obtain an initial picture; the core Processing only needs to respectively carry out downsampling on the mask picture, the initial picture and the mask picture, namely upsampling after downsampling, so that most of repairing Processing processes can be finished, a complex learning model is not needed, the operation complexity is low, all pixel values of the repaired picture can be obtained by calculating the value of each pixel point of the upsampled initial result picture and the value of the pixel point corresponding to each pixel point of the upsampled initial result picture in the upsampled mask result picture, the Processing speed is high, the repaired picture can be quickly obtained, the repairing time of the picture is greatly reduced, the picture repairing method is applied to a terminal, when the terminal is a mobile phone, only a Central Processing Unit (CPU) is used for operation, a Graphic Processing Unit (GPU) is not used for operation, 30FPS (framework processor Second, number of transmission frames per second) or higher.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

Fig. 1 is a schematic flowchart of a picture restoration method according to an embodiment of the present disclosure;

fig. 2A is a schematic diagram of a picture to be repaired according to an embodiment of the disclosure;

fig. 2B is a schematic diagram of an initial picture corresponding to the picture to be repaired in fig. 2A according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of a repair picture corresponding to the initial picture of fig. 2B according to an embodiment of the disclosure;

FIG. 4 is a detailed flowchart of step S101 in FIG. 1;

fig. 5 is a schematic structural diagram of a picture restoration device according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an electronic device for picture restoration according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing the devices, modules or units, and are not used for limiting the devices, modules or units to be different devices, modules or units, and also for limiting the sequence or interdependence relationship of the functions executed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The present disclosure provides a picture restoration method, an apparatus, a task terminal, a management terminal, and a medium, which are intended to solve the above technical problems in the prior art.

The following describes the technical solutions of the present disclosure and how to solve the above technical problems in specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present disclosure will be described below with reference to the accompanying drawings.

It should be noted that, the "terminal" used in the embodiments of the present disclosure may be a Mobile phone, a tablet computer, a PDA (Personal Digital Assistant), an MID (Mobile Internet Device), and the like.

Referring to fig. 1, an embodiment of the present disclosure provides a method for repairing a picture, which can be applied to a terminal, and the method includes:

step S101: and processing the picture to be repaired according to a preset mask picture corresponding to the picture to be repaired to obtain an initial picture.

In the embodiment of the present disclosure, the picture to be repaired may be a grayscale picture, a picture of an R channel, a picture of a G channel, a picture of a B channel, and the like. The mask picture is a template of the image filter, and which part of the picture to be repaired needs to be repaired and which part does not need to be repaired can be determined according to the mask picture. In the embodiment of the present disclosure, the mask picture is preset. It can be understood by those skilled in the art that the mask picture has the same size as the picture to be repaired. In the mask picture, the pixel value of the part needing to be repaired corresponding to the mask picture is a first preset numerical value, and the pixel values of other parts of the mask picture are second preset numerical values. Optionally, the pixel value of the portion to be repaired corresponding to the mask picture is 0, and the pixel values of other portions of the mask picture are 1.

The processing of the picture to be repaired is to process the part of the picture to be repaired, which needs to be repaired, such as changing the pixel value of the part of the picture to be repaired. And after the repaired picture is processed, obtaining an initial picture. Referring to fig. 2A and fig. 2B, fig. 2B is a schematic diagram of an initial picture obtained by processing the picture to be repaired in fig. 2A.

Step S102: and respectively downsampling the mask picture and the initial picture to obtain a downsampled lower sampling mask result picture of the mask picture and a downsampled lower sampling initial result picture of the initial picture.

As will be understood by those skilled in the art, when the mask picture is downsampled a plurality of times, the downsampling is performed next time based on the result of each downsampling. In the down-sampling, the number of times used for down-sampling is not limited, and the number of times may be 1/2 or 1/4. Specifically, when the multiple is 1/2, the side length of the picture obtained by each downsampling is half of the side length of the picture before downsampling. When the mask image is downsampled, the specific times of downsampling the mask image are not limited, and the mask image can meet the preset conditions. The preset condition may be that the number of pixels of the shortest side of the final picture obtained by downsampling the mask picture is smaller than the preset number of pixels, and the downsampling of the mask picture is performed. And downsampling the mask picture to meet the preset conditions, and obtaining the final picture which is the downsampling mask result picture.

As will be understood by those skilled in the art, when the initial picture is downsampled a plurality of times, the downsampling is performed next time based on the result of each downsampling. In the down-sampling, the multiple used for the down-sampling is not limited, and the multiple may be 2, 4, or the like. It is understood that the down-sampling is a reduction of the picture size. Specifically, when the multiple is 2, the side length of the picture obtained by each downsampling is half of the side length of the picture before downsampling, that is, the picture is reduced by 2 times. When the initial picture is downsampled, the specific times of downsampling the initial picture are not limited, and the preset condition is met. The preset condition may be that the number of pixels of the shortest side of the final picture obtained by downsampling the initial picture is smaller than a preset number of pixels, and the downsampling of the initial picture is performed. And downsampling the initial picture to meet the preset conditions, wherein the obtained final picture is a downsampling initial result picture. In the implementation of the present disclosure, the preset conditions for respectively downsampling the mask picture and the initial picture are the same, and the number of downsampling the mask picture and the number of downsampling the initial picture are the same. The size of the obtained lower sampling initial result graph is the same as that of the lower sampling mask result graph.

Step S103: and respectively performing upsampling on the lower sampling mask result graph and the lower sampling initial result graph to obtain an upsampling mask result graph after upsampling on the lower sampling mask result graph and an upsampling initial result graph after upsampling on the lower sampling initial result graph.

The number of times of upsampling the downsampling mask result picture and the number of times of downsampling the mask picture are consistent. It is understood that the upsampling is an enlargement of the picture size. And when the lower sampling mask result image is subjected to up-sampling, the multiple used for the up-sampling is consistent with the multiple used for the down-sampling of the mask image. In the embodiment of the present disclosure, the lower sampling mask result graph is up-sampled to obtain the upper sampling mask graph, the upper sampling mask graph may be used to correct the down-sampled mask picture with a size corresponding to the upper sampling mask graph, so as to be used as a basis for up-sampling again, when the number of final up-sampling times is consistent with the number of down-sampling times, the mask picture is corrected by using the upper sampling mask graph obtained by up-sampling, and the corrected picture is the upper sampling mask result graph.

The number of times the downsampling is performed on the initial result picture is consistent with the number of times the initial picture is downsampled. And when the lower sampling initial result image is subjected to up-sampling, the multiple used for the up-sampling is consistent with the multiple used for the down-sampling of the initial image. In the embodiment of the present disclosure, the lower sampling initial result graph is up-sampled to obtain an upper sampling initial result graph, the upper sampling initial graph may be used to correct the down-sampled picture of the initial picture with a size corresponding to the upper sampling initial graph, so as to be used as a basis for up-sampling again, when the final up-sampling frequency is consistent with the down-sampling frequency, the upper sampling initial graph obtained by the up-sampling is used to correct the initial picture, and the picture obtained after the correction is the upper sampling initial result graph.

Step S104: and calculating based on the value of each pixel point of the upper-sampling initial result graph and the value of the pixel point corresponding to each pixel point of the upper-sampling initial result graph in the upper-sampling mask result graph to obtain the value of each pixel point of the repair picture.

And repairing the picture, namely repairing the repaired picture to obtain the picture. The value of each pixel point of the restored picture is calculated. For example, the upper sampling initial result graph comprises an a1 pixel point, a B1 pixel point, a C1 pixel point and a D1 pixel point, and the upper sampling mask result graph comprises an a2 pixel point, a B2 pixel point, a C2 pixel point and a D2 pixel point, wherein the position of the a1 pixel point corresponds to the position of the a2 pixel point, the position of the B1 pixel point corresponds to the position of the B2 pixel point, the position of the C1 pixel point corresponds to the position of the C2 pixel point, and the position of the D1 pixel point corresponds to the position of the D2 pixel point. The repaired picture comprises an A3 pixel point corresponding to the pixel point of the A1 pixel point, a B3 pixel point corresponding to the pixel point of the B1 pixel point, a C3 pixel point corresponding to the pixel point of the C1 pixel point and a D3 pixel point corresponding to the pixel point of the D1 pixel point, when the A3 pixel point is calculated, calculation needs to be carried out based on the A1 pixel point and the A2 pixel point, and when the B3 pixel point is calculated, calculation needs to be carried out based on the B1 pixel point and the B2 pixel point; when the C3 pixel point is calculated, calculation needs to be carried out based on the C1 pixel point and the C2 pixel point; when the D3 pixel point is calculated, calculation needs to be performed based on the D1 pixel point and the D2 pixel point. Specifically, the calculation method is not limited, and optionally, the value of each pixel point in the upper sampling initial result graph may be divided by the value of the pixel point corresponding to each pixel point in the upper sampling mask result graph, so as to obtain the value of each pixel point of the repair picture, for example, the value of the pixel point A3 is the value of the pixel point a1 divided by the value of the pixel point a 2.

Referring to fig. 3, fig. 3 is a schematic diagram illustrating the picture of fig. 2B after being repaired.

According to the picture restoration method provided by the embodiment of the disclosure, according to a preset mask picture corresponding to a picture to be restored, the picture to be restored is processed to obtain an initial picture; the core Processing only needs to respectively carry out downsampling on the mask picture, the initial picture and the mask picture, namely upsampling after downsampling, so that most of repairing Processing processes can be finished, a complex learning model is not needed, the operation complexity is low, all pixel values of the repaired picture can be obtained by calculating the value of each pixel point of the upsampled initial result picture and the value of the pixel point corresponding to each pixel point of the upsampled initial result picture in the upsampled mask result picture, the Processing speed is high, the repaired picture can be quickly obtained, the repairing time of the picture is greatly reduced, the picture repairing method is applied to a terminal, when the terminal is a mobile phone, only a Central Processing Unit (CPU) is used for operation, a Graphic Processing Unit (GPU) is not used for operation, 30FPS (frame Per Second, number of transmission frames per second) or higher.

Referring to fig. 4, optionally, processing the picture to be repaired to obtain an initial picture according to a preset mask picture corresponding to the picture to be repaired includes:

s401: and determining the part to be repaired of the picture to be repaired according to the mask picture.

In the embodiment of the disclosure, in the mask picture, the pixel value of the portion to be repaired corresponding to the mask picture is 0, and the pixel values of other portions of the mask picture are 1. And determining the mask picture, wherein the part to be repaired of the picture to be repaired corresponding to the mask picture can also be determined.

S402: setting the pixel value of the part to be repaired of the picture to be repaired as a first preset value to obtain an initial picture.

Setting the pixel value of the part to be repaired of the picture to be repaired to be a first preset value, and not changing the pixel values of other parts of the picture to be repaired. In the embodiment of the present disclosure, the pixel value of the portion to be repaired of the picture to be repaired is the same as the pixel value of the portion to be repaired corresponding to the mask picture, and both the pixel values are 0.

The pixel value of the part to be repaired of the picture to be repaired is set to be the first preset value, so that the subsequent processing of the initial picture can be facilitated, and the repaired picture can be quickly obtained.

Optionally, the downsampling is performed on the mask picture and the initial picture respectively to obtain a downsampled mask result graph of the mask picture and a downsampled initial result graph of the initial picture, and the downsampling includes:

respectively carrying out downsampling on the mask picture and the initial picture according to a preset multiple to obtain a first intermediate picture after the downsampling of the mask picture and a second intermediate picture after the downsampling of the initial picture; taking the first intermediate image as a mask image and taking the second intermediate image as an initial image;

repeatedly executing downsampling on the mask picture and the initial picture according to a preset multiple to obtain a first intermediate picture after downsampling of the mask picture and a second intermediate picture after downsampling of the initial picture; and taking the first intermediate image as a mask image and the second intermediate image as an initial image, and determining the first intermediate image as a downsampled mask result image of the mask image and determining the second intermediate image as a downsampled initial result image of the initial image when the number of pixels of the shortest side of the first intermediate image and the second intermediate image is less than the preset number of pixels.

The preset multiple is not limited, for example, the preset multiple may be 2, 4, etc. Specifically, when the preset multiple is 2, the mask image is downsampled, and the side length of the first intermediate map obtained by downsampling is one half of the side length of the mask image before downsampling, that is, the side length is reduced by 2 times.

The specific number of the preset pixels is not limited, and may be 4, 8, 10, and the like, and in the embodiment of the present disclosure, the number of the preset pixels is 4. And when the number of the pixels on the shortest side of the first intermediate image is less than 4, determining the first intermediate image as a down-sampling mask result image after down-sampling of the mask image. And when the number of the shortest edge pixels of the second intermediate image is less than 4, determining the second intermediate image as a downsampling initial result image of the initial image. And the mask picture and the initial picture have the same size, the mask picture is downsampled, the initial picture is downsampled, the adopted preset times are the same, and the finally obtained lower-sampling mask result picture and the finally obtained lower-sampling initial result picture have the same size.

Optionally, the upsampling is performed on the lower sampling mask result graph and the lower sampling initial result graph respectively, so as to obtain an upsampling mask result graph after the upsampling is performed on the lower sampling mask result graph and an upsampling initial result graph after the upsampling is performed on the lower sampling initial result graph, and the upsampling includes:

respectively performing up-sampling on the lower sampling mask result graph and the lower sampling initial result graph according to preset multiples to obtain an upper sampling mask graph after the up-sampling of the lower sampling mask result graph and an upper sampling initial graph after the up-sampling of the lower sampling initial result graph;

correcting a lower sampling mask result image or a mask image with the same size as the upper sampling mask image according to the upper sampling mask image and preset weight to obtain a third intermediate image, and taking the third intermediate image as a lower sampling mask result image needing to be subjected to upper sampling; correcting a lower sampling initial result image or an initial image with the same size as the upper sampling initial image according to the upper sampling initial image and preset weight to obtain a fourth intermediate image, and taking the fourth intermediate image as the lower sampling initial result image needing to be subjected to upper sampling;

repeatedly performing upsampling on the lower sampling mask result graph and the lower sampling initial result graph according to preset multiples to obtain an upsampling mask graph after upsampling on the lower sampling mask result graph and an upsampling initial graph after upsampling on the lower sampling initial result graph; correcting a lower sampling mask result image or a mask image with the same size as the upper sampling mask image according to the upper sampling mask image and preset weight to obtain a third intermediate image, and taking the third intermediate image as a lower sampling mask result image needing to be subjected to upper sampling; and correcting the lower sampling initial result graph or the initial picture with the same size as the upper sampling initial graph according to the upper sampling initial graph and preset weight to obtain a fourth intermediate graph, taking the fourth intermediate graph as the lower sampling initial result graph needing to be subjected to the upper sampling, and determining the third intermediate graph as the upper sampling mask result graph after the upper sampling of the lower sampling mask result graph and the fourth intermediate graph as the upper sampling initial result graph after the upper sampling of the lower sampling initial result graph when the times of the upper sampling and the times of the lower sampling are consistent.

In the embodiment of the present disclosure, the preset multiple is not limited, for example, the preset multiple may be 2, 4, and the like. Specifically, when the preset multiple is 2, the lower sampling mask result graph is subjected to down sampling, and the side length of the upper sampling mask graph obtained by up sampling is 2 times of the side length of the lower sampling mask result graph before the down sampling. And if one side length of the lower sampling mask result graph is 2, the side length of the upper sampling mask graph corresponding to the side length, which is obtained by performing up-sampling on the lower sampling mask result graph, is 4. The multiple used for up-sampling the down-sampling mask result image, the multiple used for up-sampling the down-sampling initial result image and the multiple used for down-sampling the mask image are the same. The weight can be set according to the requirement, and the embodiment of the disclosure is not limited. In the embodiment of the present disclosure, the weight may be set to 0.001-1.2, 0.01-1, etc.

When the picture with the same size as the upper sampling mask picture is corrected according to the upper sampling mask picture and the preset weight, the correction after the last upper sampling is the correction of the mask picture, and the correction before the last upper sampling is the correction of the lower sampling mask result picture with the same size as the upper sampling mask picture.

Specifically, if the mask image is downsampled for the 4 th time according to the preset multiple, and the number of pixels of the shortest side of the obtained image is less than 4, the downsampling is only performed for 4 times, wherein a first middle graph a1, a first middle graph a2, a first middle graph a3, and an upsampling mask result graph a4 are sequentially obtained in the downsampling process, when the upsampling is performed, the upsampling mask result graph a4 is upsampled to obtain an upsampling mask graph d1, the first middle graph a3 with the same size as the upsampling mask graph d1 is corrected according to the upsampling mask graph d1 and preset weight to obtain a third middle graph d1, and the third middle graph is used as a downsampling mask result graph needing to be upsampled; and sequentially performing upsampling, performing upsampling for the 4 th time, and correcting the mask picture to an upsampling mask picture d4 according to the upsampling mask picture d4 and preset weight to obtain a third intermediate picture d4, wherein the size of the third intermediate picture is the same as that of the mask picture, and the third intermediate picture d4 is determined as the upsampled upsampling mask result picture of the lower upsampling mask result picture. Wherein, the correction after the 1 st upsampling, the correction after the 2 nd upsampling and the correction after the 3 rd upsampling are the correction of the first middle graph with the same size as the upsampling mask graph. When the correction is performed, each pixel point is corrected.

And performing upsampling on the lower sampling initial result graph to finally obtain an upper sampling initial result graph, wherein the step of performing upsampling on the lower sampling mask result graph to finally obtain an upper sampling mask result graph is consistent with that of performing upsampling on the lower sampling mask result graph, and detailed description is not repeated in the embodiment of the disclosure.

Optionally, the step of correcting the lower sampling mask result image or the mask image with the same size as the upper sampling mask image according to the upper sampling mask image and the preset weight to obtain a third intermediate image includes:

respectively obtaining the value of each pixel point in an upper sampling mask image and a lower sampling mask result image or mask image with the same size as the upper sampling mask image;

according to the value of each pixel point of the upper sampling mask image and the preset weight, the value of each corresponding pixel point in the lower sampling mask result image or the mask image with the same size as the upper sampling mask image is corrected based on the following formula:

M＝m1+w×m2；

wherein M1 is the value of the pixel point in the lower sampling mask result picture or the mask picture with the same size as the upper sampling mask picture, M2 is the value of the pixel point in the upper sampling mask picture, the position of M1 corresponds to the position of M2, w is the weight, and M is the value of the pixel point of the third middle picture.

When the values of the pixel points of the third intermediate graph are calculated, the value of each pixel point of the third intermediate graph is calculated, the pixel point in the upper sampling mask graph is m2, then m2 is multiplied by the weight w to obtain w × m2, then the values of the pixel points corresponding to the pixel points in the upper sampling mask graph in the upper sampling mask result graph or the upper sampling mask graph in the mask graph are added, and the obtained pixel values are the values of the pixel points of the third intermediate graph.

Optionally, modifying a lower sampling initial result graph or an initial picture with the same size as the upper sampling initial graph according to the upper sampling initial graph and a preset weight to obtain a fourth intermediate graph, including:

respectively obtaining the value of each pixel point in an upper acquisition initial image and a lower acquisition initial result image or initial image with the same size as the upper acquisition initial image;

according to the value of each pixel point of the upper-sampling initial image and the preset weight, correcting the value of each corresponding pixel point in the lower-sampling initial result image or the initial image with the same size as the upper-sampling initial image based on the following formula:

H＝h1+w×h2；

h1 is the value of a pixel point in the lower sampling initial result graph or the initial graph with the same size as the upper sampling initial graph, H2 is the value of the pixel point in the upper sampling initial graph, the position of H1 corresponds to the position of H2, w is the weight, and H is the value of the pixel point of the fourth intermediate graph.

When calculating the values of the pixel points of the fourth intermediate graph, the value of each pixel point of the fourth intermediate graph is calculated, the pixel point in the upper-sampling initial graph is h2, then h2 is multiplied by the weight w to obtain w × h2, then the value of the pixel point corresponding to the pixel point in the upper-sampling initial graph in the upper-sampling initial result graph or the upper-sampling initial graph is added, and the obtained pixel value is the value of the pixel point of the fourth intermediate graph.

Optionally, when the picture to be repaired is an RGB three-channel picture, before the picture to be repaired is processed to obtain the initial picture, the picture repairing method further includes:

and processing the RGB three-channel picture to obtain a first picture to be repaired of the R channel, a second picture to be repaired of the G channel and a third picture to be repaired of the B channel.

The RGB three-channel picture, that is, the color picture, needs to be processed to obtain a picture corresponding to each channel.

When the picture to be restored is an RGB three-channel picture, processing the picture to be restored according to a preset mask picture corresponding to the picture to be restored to obtain an initial picture, wherein the processing method comprises the following steps:

and respectively repairing the first picture to be repaired, the second picture to be repaired and the third picture to be repaired according to the mask pictures corresponding to the first picture to be repaired, the second picture to be repaired and the third picture to be repaired.

The first picture to be repaired, the second picture to be repaired and the third picture to be repaired are from the same RGB three-channel picture, and the sizes of the first picture to be repaired, the second picture to be repaired and the third picture to be repaired are consistent with the positions to be repaired. The first picture to be repaired, the second picture to be repaired and the third picture to be repaired correspond to the same mask picture. The first picture to be repaired is repaired to obtain a first repaired picture, the second picture to be repaired is repaired to obtain a second repaired picture, and the third picture to be repaired is repaired to obtain a third repaired picture.

Optionally, when obtaining a first repair picture corresponding to the first picture to be repaired, a second repair picture corresponding to the second picture to be repaired, and a third repair picture corresponding to the third picture to be repaired, the picture repairing method further includes:

and synthesizing the first repair picture, the second repair picture and the third repair picture to obtain the color repair picture.

The first repair picture is an R channel picture, the second repair picture is a G channel picture, and the third repair picture is a B channel picture. And when the first repairing picture, the second repairing picture and the third repairing picture are synthesized, the color repairing picture corresponding to the picture to be repaired can be obtained. It can be understood that when the picture to be repaired is a gray-scale picture, the obtained picture is the gray-scale repaired picture.

Referring to fig. 5, an embodiment of the present disclosure provides a picture restoration device 50, where the picture restoration device 50 is applied to a terminal, and the picture restoration device can implement the picture restoration method of the embodiment, and the picture restoration device 50 may include: a pre-processing module 501, a down-sampling processing module 502, an up-sampling processing module 503, and a repair module 504, wherein,

the pre-processing module 501 is configured to process the picture to be repaired to obtain an initial picture according to a preset mask picture corresponding to the picture to be repaired;

a down-sampling processing module 502, configured to perform down-sampling on the mask picture and the initial picture, respectively, to obtain a down-sampling mask result graph after the mask picture is down-sampled and a down-sampling initial result graph after the initial picture is down-sampled;

an upsampling processing module 503, configured to perform upsampling on the lower sampling mask result graph and the lower sampling initial result graph, respectively, to obtain an upsampled upper sampling mask result graph of the lower sampling mask result graph and an upsampled upper sampling initial result graph of the lower sampling initial result graph;

the repairing module 504 is configured to perform calculation based on the value of each pixel point of the upper sampling initial result graph and the value of a pixel point corresponding to each pixel point of the upper sampling initial result graph in the upper sampling mask result graph, so as to obtain a value of each pixel point of the repairing picture.

According to the picture restoration device provided by the embodiment of the disclosure, according to the preset mask picture corresponding to the picture to be restored, the picture to be restored is processed to obtain an initial picture; the core Processing only needs to respectively carry out downsampling on the mask picture, the initial picture and the mask picture, namely upsampling after downsampling, so that most of repairing Processing processes can be finished, a complex learning model is not needed, the operation complexity is low, all pixel values of the repaired picture can be obtained by calculating the value of each pixel point of the upsampled initial result picture and the value of the pixel point corresponding to each pixel point of the upsampled initial result picture in the upsampled mask result picture, the Processing speed is high, the repaired picture can be quickly obtained, the repairing time of the picture is greatly reduced, the picture repairing device is applied to a terminal, when the terminal is a mobile phone, only a Central Processing Unit (CPU) is used for operation, a Graphic Processing Unit (GPU) is not used for operation, 30FPS (frame Per Second, number of transmission frames per second) or higher.

Optionally, the preprocessing module 501 includes:

the determining unit is used for determining the part to be repaired of the picture to be repaired according to the mask picture;

and the second determining unit is used for setting the pixel value of the part to be repaired of the picture to be repaired to be a first preset value so as to obtain the initial picture.

Optionally, the downsampling processing module 502 includes:

the down-sampling processing unit is used for respectively down-sampling the mask picture and the initial picture according to preset multiples to obtain a first intermediate picture after down-sampling of the mask picture and a second intermediate picture after down-sampling of the initial picture; taking the first intermediate image as a mask image and taking the second intermediate image as an initial image;

the first repeated execution unit is used for repeatedly executing downsampling on the mask picture and the initial picture according to preset multiples to obtain a first intermediate picture after downsampling of the mask picture and a second intermediate picture after downsampling of the initial picture; and taking the first intermediate image as a mask image and the second intermediate image as an initial image, and determining the first intermediate image as a downsampled mask result image of the mask image and determining the second intermediate image as a downsampled initial result image of the initial image when the number of pixels of the shortest side of the first intermediate image and the second intermediate image is less than the preset number of pixels.

Optionally, the upsampling processing module 503 includes:

the upper sampling processing unit is used for respectively carrying out upper sampling on the lower sampling mask result graph and the lower sampling initial result graph according to preset multiples to obtain an upper sampling mask graph after the upper sampling of the lower sampling mask result graph and an upper sampling initial graph after the upper sampling of the lower sampling initial result graph;

the correction unit is used for correcting the lower sampling mask result image or the mask image with the same size as the upper sampling mask image according to the upper sampling mask image and the preset weight to obtain a third intermediate image, and the third intermediate image is used as the lower sampling mask result image needing to be subjected to the upper sampling; correcting a lower sampling initial result image or an initial image with the same size as the upper sampling initial image according to the upper sampling initial image and preset weight to obtain a fourth intermediate image, and taking the fourth intermediate image as the lower sampling initial result image needing to be subjected to upper sampling;

the second repeated execution unit is used for repeatedly executing the up-sampling on the lower-sampling mask result graph and the lower-sampling initial result graph according to preset multiples to obtain an up-sampling mask graph after the up-sampling of the lower-sampling mask result graph and an up-sampling initial graph after the up-sampling of the lower-sampling initial result graph; correcting a lower sampling mask result image or a mask image with the same size as the upper sampling mask image according to the upper sampling mask image and preset weight to obtain a third intermediate image, and taking the third intermediate image as a lower sampling mask result image needing to be subjected to upper sampling; and correcting the lower sampling initial result graph or the initial picture with the same size as the upper sampling initial graph according to the upper sampling initial graph and preset weight to obtain a fourth intermediate graph, taking the fourth intermediate graph as the lower sampling initial result graph needing to be subjected to the upper sampling, and determining the third intermediate graph as the upper sampling mask result graph after the upper sampling of the lower sampling mask result graph and the fourth intermediate graph as the upper sampling initial result graph after the upper sampling of the lower sampling initial result graph when the times of the upper sampling and the times of the lower sampling are consistent.

Optionally, the correction unit comprises:

the first acquisition unit is used for respectively acquiring the values of each pixel point in the upper sampling mask image and the lower sampling mask result image or the mask image with the same size as the upper sampling mask image;

the mask correction unit is used for correcting the value of each pixel point in the lower sampling mask result graph or the mask picture with the same size as the upper sampling mask graph based on the following formula according to the value and the preset weight of each pixel point of the upper sampling mask graph:

M＝m1+w×m2；

wherein M1 is the value of a pixel point in the lower sampling mask result picture or the mask picture with the same size as the upper sampling mask picture, M2 is the value of a pixel point in the upper sampling mask picture, the position of M1 corresponds to the position of M2, w is a weight, and M is the value of a pixel point of a third middle picture;

the second acquisition unit is used for respectively acquiring the values of each pixel point in the upper acquisition initial image and the lower acquisition initial result image or the initial image with the same size as the upper acquisition initial image;

the initial correction unit is used for correcting the value of each pixel point in the lower-sampling initial result graph or the initial picture with the same size as the upper-sampling initial graph based on the following formula according to the value and the preset weight of each pixel point in the upper-sampling initial graph:

H＝h1+w×h2；

Optionally, when the picture to be restored is an RGB three-channel picture, the picture restoring apparatus 30 further includes:

and the separation module is used for processing the RGB three-channel picture to obtain a first picture to be repaired of the R channel, a second picture to be repaired of the G channel and a third picture to be repaired of the B channel.

The preprocessing module 501 is further configured to respectively repair the first to-be-repaired picture, the second to-be-repaired picture, and the third to-be-repaired picture according to the mask pictures corresponding to the first to-be-repaired picture, the second to-be-repaired picture, and the third to-be-repaired picture.

When obtaining a first repair picture corresponding to the first picture to be repaired, a second repair picture corresponding to the second picture to be repaired, and a third repair picture corresponding to the third picture to be repaired, the picture repairing apparatus 50 further includes:

and the synthesis module is used for synthesizing the first repair picture, the second repair picture and the third repair picture to obtain the color repair picture.

Referring to fig. 6, a schematic diagram of an electronic device 600 suitable for implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

The electronic device includes: a memory and a processor, wherein the processor may be referred to as the processing device 601 hereinafter, and the memory may include at least one of a Read Only Memory (ROM)602, a Random Access Memory (RAM)603 and a storage device 608 hereinafter, which are specifically shown as follows:

as shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText transfer protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: processing the picture to be repaired according to a preset mask picture corresponding to the picture to be repaired to obtain an initial picture; respectively carrying out downsampling on the mask picture and the initial picture to obtain a downsampled lower sampling mask result picture of the mask picture and a downsampled lower sampling initial result picture of the initial picture; respectively performing up-sampling on the lower sampling mask result graph and the lower sampling initial result graph to obtain an upper sampling mask result graph after the up-sampling of the lower sampling mask result graph and an upper sampling initial result graph after the up-sampling of the lower sampling initial result graph; and calculating based on the value of each pixel point of the upper-sampling initial result graph and the value of the pixel point corresponding to each pixel point of the upper-sampling initial result graph in the upper-sampling mask result graph to obtain the value of each pixel point of the repair picture.

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules or units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a module or unit does not in some cases constitute a limitation of the unit itself, for example, an upsampling processing module may also be described as a "unit that performs upsampling and modification".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, there is provided a picture restoration method including:

According to one or more embodiments of the present disclosure, downsampling a mask picture and an initial picture respectively to obtain a downsampled mask result graph of the mask picture and a downsampled initial result graph of the initial picture, includes:

According to one or more embodiments of the present disclosure, processing a to-be-repaired picture according to a preset mask picture corresponding to the to-be-repaired picture to obtain an initial picture includes:

determining a part to be repaired of the picture to be repaired according to the mask picture;

setting the pixel value of the part to be repaired of the picture to be repaired as a first preset value to obtain an initial picture.

According to one or more embodiments of the present disclosure, the upsampling is performed on the lower sampling mask result graph and the lower sampling initial result graph respectively to obtain an upsampled upper sampling mask result graph of the lower sampling mask result graph and an upsampled upper sampling initial result graph of the lower sampling initial result graph, including:

According to one or more embodiments of the present disclosure, modifying a lower sampling mask result graph or a mask graph having the same size as an upper sampling mask graph according to the upper sampling mask graph and a preset weight to obtain a third middle graph includes:

M＝m1+w×m2；

correcting a lower sampling initial result image or an initial image with the same size as the upper sampling initial image according to the upper sampling initial image and preset weight to obtain a fourth intermediate image, wherein the fourth intermediate image comprises the following steps:

H＝h1+w×h2；

According to one or more embodiments of the present disclosure, before processing the picture to be restored to obtain the initial picture when the picture to be restored is an RGB three-channel picture, the method further includes:

processing the RGB three-channel picture to obtain a first picture to be repaired of an R channel, a second picture to be repaired of a G channel and a third picture to be repaired of a B channel;

according to one or more embodiments of the present disclosure, processing a picture to be repaired to obtain an initial picture according to a preset mask picture corresponding to the picture to be repaired includes:

According to one or more embodiments of the present disclosure, when obtaining a first repair picture corresponding to a first picture to be repaired, a second repair picture corresponding to a second picture to be repaired, and a third repair picture corresponding to a third picture to be repaired, the method further includes:

According to one or more embodiments of the present disclosure, there is provided a picture restoration apparatus including:

According to one or more embodiments of the present disclosure, a preprocessing module includes:

According to one or more embodiments of the present disclosure, a downsampling processing module includes:

According to one or more embodiments of the present disclosure, an up-sampling processing module includes:

According to one or more embodiments of the present disclosure, the correction unit includes:

M＝m1+w×m2；

H＝h1+w×h2；

According to one or more embodiments of the present disclosure, the preprocessing module is further configured to respectively repair the first to-be-repaired picture, the second to-be-repaired picture, and the third to-be-repaired picture according to the mask pictures corresponding to the first to-be-repaired picture, the second to-be-repaired picture, and the third to-be-repaired picture.

According to one or more embodiments of the present disclosure, when obtaining a first repair picture corresponding to a first picture to be repaired, a second repair picture corresponding to a second picture to be repaired, and a third repair picture corresponding to a third picture to be repaired, the picture repairing apparatus further includes:

According to one or more embodiments of the present disclosure, there is provided an electronic device including:

one or more processors;

a memory;

one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the picture restoration method according to the above-described embodiment is performed.

According to one or more embodiments of the present disclosure, there is provided a computer-readable medium on which a computer program is stored, which when executed by a processor, implements the picture restoration method of any of the above-described embodiments.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A picture restoration method is characterized by comprising the following steps:

2. The method according to claim 1, wherein the downsampling the mask picture and the initial picture to obtain a downsampled mask result graph of the mask picture and a downsampled initial result graph of the initial picture comprises:

repeatedly executing the downsampling of the mask picture and the initial picture according to the preset multiple to obtain a first intermediate picture after the downsampling of the mask picture and a second intermediate picture after the downsampling of the initial picture; and taking the first intermediate image as a mask image and the second intermediate image as an initial image, and determining the first intermediate image as a downsampled mask result image of the mask image and determining the second intermediate image as a downsampled initial result image of the initial image when the number of pixels of the shortest side of the first intermediate image and the second intermediate image is less than the preset number of pixels.

3. The picture restoration method according to claim 1, wherein the pixel value of the portion to be restored corresponding to the mask picture is a first predetermined value, the pixel values of the other portions of the mask picture are second predetermined values, and the processing of the picture to be restored according to the preset mask picture corresponding to the picture to be restored to obtain the initial picture comprises:

and setting the pixel value of the part to be repaired of the picture to be repaired as a first preset value to obtain an initial picture.

4. The method according to claim 1, wherein the upsampling is performed on the lower sampling mask result graph and the lower sampling initial result graph respectively to obtain an upsampled upper sampling mask result graph of the lower sampling mask result graph and an upsampled upper sampling initial result graph of the lower sampling initial result graph, and the upsampling comprises:

correcting a lower sampling mask result image or a mask image with the same size as the upper sampling mask image according to the upper sampling mask image and preset weight to obtain a third intermediate image, and taking the third intermediate image as a lower sampling mask result image needing to be subjected to upper sampling; correcting a lower sampling initial result image or an initial image with the same size as the upper sampling initial image according to the upper sampling initial image and a preset weight to obtain a fourth intermediate image, and taking the fourth intermediate image as the lower sampling initial result image needing to be subjected to upper sampling;

repeatedly performing the up-sampling on the lower sampling mask result graph and the lower sampling initial result graph according to the preset multiple to obtain an up-sampling mask graph after the up-sampling of the lower sampling mask result graph and an up-sampling initial graph after the up-sampling of the lower sampling initial result graph; correcting a lower sampling mask result image or a mask image with the same size as the upper sampling mask image according to the upper sampling mask image and preset weight to obtain a third intermediate image, and taking the third intermediate image as a lower sampling mask result image needing to be subjected to upper sampling; and correcting a lower sampling initial result graph or an initial picture with the same size as the upper sampling initial graph according to the upper sampling initial graph and preset weight to obtain a fourth intermediate graph, and taking the fourth intermediate graph as the lower sampling initial result graph needing to be subjected to the upper sampling until the times of the upper sampling and the lower sampling are consistent, determining the third intermediate graph as the upper sampling mask result graph after the upper sampling of the lower sampling mask result graph and determining the fourth intermediate graph as the upper sampling initial result graph after the upper sampling of the lower sampling initial result graph.

5. The method according to claim 4, wherein the correcting the lower sampling mask result image or the mask image with the same size as the upper sampling mask image according to the upper sampling mask image and the preset weight to obtain a third intermediate image comprises:

according to the value and the preset weight of each pixel point of the upper sampling mask image, correcting the value of each corresponding pixel point in the lower sampling mask result image or the mask image with the same size as the upper sampling mask image based on the following formula:

M＝m1+w×m2；

wherein M1 is the value of a pixel point in a lower sampling mask result graph or a mask graph with the same size as the upper sampling mask graph, M2 is the value of a pixel point in the upper sampling mask graph, the position of M1 corresponds to the position of M2, w is the weight, and M is the value of a pixel point of a third middle graph;

the correcting a lower sampling initial result graph or an initial picture with the same size as the upper sampling initial graph according to the upper sampling initial graph and a preset weight to obtain a fourth intermediate graph comprises the following steps:

H＝h1+w×h2；

the H1 is the value of a pixel point in a lower sampling initial result graph or an initial graph with the same size as the upper sampling initial graph, the H2 is the value of a pixel point in the upper sampling initial graph, the position of H1 corresponds to the position of H2, w is the weight, and H is the value of a pixel point of a fourth intermediate graph.

6. The method according to claim 1, wherein when the picture to be repaired is an RGB three-channel picture, before the processing the picture to be repaired to obtain an initial picture, the method further comprises:

the processing the picture to be repaired according to the preset mask picture corresponding to the picture to be repaired to obtain an initial picture comprises the following steps:

and repairing the first picture to be repaired, the second picture to be repaired and the third picture to be repaired respectively according to the mask pictures corresponding to the first picture to be repaired, the second picture to be repaired and the third picture to be repaired.

7. The picture restoration method according to claim 6, wherein when obtaining a first restoration picture corresponding to the first picture to be restored, a second restoration picture corresponding to the second picture to be restored, and a third restoration picture corresponding to the third picture to be restored, the method further comprises:

and synthesizing the first repair picture, the second repair picture and the third repair picture to obtain a color repair picture.

8. A picture restoration device, comprising:

the device comprises a preprocessing module, a storage module and a processing module, wherein the preprocessing module is used for processing a picture to be repaired according to a preset mask picture corresponding to the picture to be repaired to obtain an initial picture;

and the restoration module is used for calculating based on the value of each pixel point of the upper acquisition initial result graph and the value of the pixel point corresponding to each pixel point of the upper acquisition initial result graph in the upper acquisition mask result graph so as to obtain the value of each pixel point of the restoration picture.

9. An electronic device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: -performing the picture restoration method according to any of claims 1-7.

10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the picture restoration method according to any one of claims 1 to 7.