CN112419179A

CN112419179A - Method, device, equipment and computer readable medium for repairing image

Info

Publication number: CN112419179A
Application number: CN202011299862.0A
Authority: CN
Inventors: 李华夏
Original assignee: Beijing Zitiao Network Technology Co Ltd
Current assignee: Beijing Zitiao Network Technology Co Ltd
Priority date: 2020-11-18
Filing date: 2020-11-18
Publication date: 2021-02-26

Abstract

The embodiment of the disclosure discloses a method, a device, an electronic device and a computer readable medium for repairing an image. One embodiment of the method comprises: preprocessing an image to be restored to obtain a first image, wherein the image to be restored is an image with missing or unobvious image features; determining at least one target region in the first image; processing the target area of at least one target area to obtain a target modification area; constructing a second image based on at least one target modification area corresponding to at least one target area; and performing image enhancement on the second image to obtain a target image. The embodiment realizes the restoration of the image with the missing or unobvious image characteristics, and ensures that the restored image content is more complete and the image quality is better by reasonably applying the restoration technology.

Description

Method, device, equipment and computer readable medium for repairing image

Technical Field

Embodiments of the present disclosure relate to the field of image processing, and in particular, to a method, an apparatus, a device, and a computer-readable medium for repairing an image.

Background

During the acquisition, transmission and storage of the image, image features may be lost or not apparent due to various factors, such as electromagnetic interference. Portions of image detail may be ignored in the process of globally inpainting an image. And the related image inpainting techniques do not solve this problem well.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Some embodiments of the present disclosure propose a method, apparatus, device and computer readable medium for repairing an image to solve the technical problems mentioned in the background section above.

In a first aspect, some embodiments of the present disclosure provide a method of repairing an image, the method comprising: preprocessing an image to be restored to obtain a first image, wherein the image to be restored is an image with missing or unobvious image features; determining at least one target region in the first image; processing the target area of at least one target area to obtain a target modification area; constructing a second image based on at least one target modification area corresponding to at least one target area; and performing image enhancement on the second image to obtain a target image.

In a second aspect, some embodiments of the present disclosure provide an apparatus for repairing an image, the apparatus comprising: the image restoration device comprises a preprocessing unit, a storage unit and a processing unit, wherein the preprocessing unit is configured to preprocess an image to be restored to obtain a first image, and the image to be restored is an image with missing or unobvious image features; a determination unit configured to determine at least one target region in the first image; the processing unit is configured to process a target area of at least one target area to obtain a target modification area; a construction unit configured to construct a second image based on at least one target modification region corresponding to the at least one target region; and the enhancement unit is configured to perform image enhancement on the second image to obtain a target image.

In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method of repairing an image as in the first aspect.

In a fourth aspect, some embodiments of the disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of repairing an image as in the first aspect.

One of the above-described various embodiments of the present disclosure has the following advantageous effects: firstly, preprocessing an image to be repaired to obtain a first image. In this step, the whole image is repaired, and the image repairing effect can be integrally grasped. Since repair of the full map may result in loss of local detail, detail repair is required for the local. Then, at least one target area in the first image is determined, and a target modification area of each target area is obtained, so that further repair of the local image is realized, and the specific details of the image can be enhanced. And finally, performing image enhancement on the second image constructed in the target modification area to obtain a target image, thereby realizing the supplement and enhancement of the image characteristics of the image to be restored. The embodiment realizes the restoration of the image with the missing or unobvious image characteristics, and ensures that the restored image content is more complete and the image quality is better by reasonably applying the restoration technology.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

FIG. 1 is a schematic illustration of one application scenario of a method of inpainting an image according to some embodiments of the present disclosure;

FIG. 2 is a flow diagram of some embodiments of a method of repairing an image according to the present disclosure;

FIG. 3 is a flow diagram of further embodiments of a method of inpainting an image according to the present disclosure;

FIG. 4 is a flow diagram of still further embodiments of a method of inpainting an image according to the present disclosure;

FIG. 5 is a schematic block diagram of some embodiments of an apparatus for inpainting an image according to the present disclosure;

FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

FIG. 1 is a schematic diagram of one application scenario of a method of inpainting an image according to some embodiments of the present disclosure.

In the application scenario of fig. 1, first, the electronic device 101 may receive an image to be repaired 102. Then, the electronic device 101 preprocesses the image to be repaired 102 to obtain a first image 103. The first image 103 is a globally repaired image. Further processing of the first image 103 is required. Next, a target region 104 in the first image 103 is determined. The target area 104 is processed to obtain a target modified area 105. A second image 106 is constructed based on the target modification area 105. Thus, the local details of the image are processed more finely. Finally, the second image 106 is subjected to image enhancement to obtain a target image 107. Image enhancement is to further enhance image features.

The electronic device 101 may be hardware or software. When the electronic device is hardware, the electronic device may be implemented as a distributed cluster formed by a plurality of servers or terminal devices, or may be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it may be installed in the above-listed hardware devices. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.

It should be understood that the number of electronic devices in fig. 1 is merely illustrative. There may be any number of electronic devices, as desired for implementation.

With continued reference to fig. 2, a flow 200 of some embodiments of a method of repairing an image according to the present disclosure is shown. The method for repairing the image comprises the following steps:

step 201, preprocessing an image to be restored to obtain a first image.

In some embodiments, an executing body (e.g., the electronic device 101 shown in fig. 1) of the method of repairing an image may receive the image to be repaired by a wired connection manner or a wireless connection manner. The image to be restored may be an image with missing or insignificant image features. It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G/5G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a uwb (ultra wideband) connection, and other wireless connection means now known or developed in the future.

In some embodiments, the image to be repaired is generally an image with some necessary image features missing or with less obvious image features. The image to be restored may be an image obtained by an old photographing apparatus or an image obtained by scanning a previous old photograph. The image to be restored may be any image. As an example, the image to be restored may be an image of a piglet, a sparrow, a house, or the like, on which a human face is displayed.

In some embodiments, the preprocessing process is an overall repair of the image to be repaired. As an example, the image to be repaired may be repaired by existing image repair software.

At least one target region in the first image is determined, step 202.

In some embodiments, based on the first image in step 201, the executing subject (e.g., the electronic device shown in fig. 1) may determine at least one target region in the first image. Generally, a target object image exists within the target area. The subject may determine the target area in the first image by target detection.

Step 203, for the target area of the at least one target area, processing the target area to obtain a target modification area.

In some embodiments, the subject may obtain the target modified region by performing image processing operations such as adjusting contrast, sharpening, etc. on the target region, as examples. Therefore, the details of the target area are restored in a targeted manner, and the image characteristic restoration of the local image is realized.

Step 204, constructing a second image based on at least one target modification area corresponding to at least one target area.

In some embodiments, the first image of the target region, which is subjected to the detail processing again, is the resulting second image.

And step 205, performing image enhancement on the second image to obtain a target image.

In some embodiments, image enhancement may be achieved by several algorithms: linear gray scale transformation, histogram equalization transformation, homomorphic filtering and other algorithms. The image enhancement is to further strengthen the image features so that the image details are clearer.

According to the method provided by some embodiments of the present disclosure, the purpose of the preprocessing operation on the image to be restored is to perform image restoration on the image to be restored within a full image range, and the obtained first image mainly considers the overall effect of the image. Then, the details of the target area are restored in a targeted manner. Finally, image enhancement is carried out, the aim of which is to improve the visual effect of the image and purposefully emphasize the overall or local characteristics of the image for the application of a given image. The original unclear image is made clear or some specific image characteristics are emphasized, the difference between different object characteristics in the image is enlarged, and uninteresting characteristics are inhibited. Therefore, the image quality can be improved, the information content is enriched, the image interpretation and recognition effects are enhanced, and the requirements of certain special analysis are met. The image with the missing or unobvious image characteristics is repaired in the above mode, and the repaired image content is more complete and the image quality is better by reasonably applying the repairing technology.

With further reference to FIG. 3, a flow 300 of further embodiments of a method of inpainting an image is illustrated. The flow 300 of the method for repairing an image comprises the following steps:

step 301, inputting an image to be restored into a preprocessing model to obtain a first image.

In some embodiments, as an example, the algorithm of the pre-processing model may include: diffusion-based methods, texture synthesis-based methods, data-driven-based image inpainting methods, and the like.

In an alternative implementation of some embodiments, the pre-processing model is obtained by: the method comprises the steps of obtaining a plurality of sample images and a sample target image corresponding to each sample image in the plurality of sample images, wherein the sample images are images with missing or unobvious image features, and the sample target images are images with complete image features corresponding to the sample images; and taking each sample image in the plurality of sample images as input, taking a sample target image corresponding to each sample image in the plurality of sample images as output, and training to obtain the preprocessing model.

In an alternative implementation of some embodiments, the sample target image is obtained by: identifying a sample target object within a sample image; adding colors based on the sample target object to obtain a sample color image; carrying out color balance processing on the sample color image to obtain a sample color balance image; carrying out defogging treatment on the sample color balance image to obtain a sample defogged image; adjusting the definition of the defogged image of the sample to obtain a clear image of the sample; and increasing pixels of the clear sample image to obtain a sample target image corresponding to the sample image. As an example, adding color to the sample image may be done by online software; color balancing of the color image can also be accomplished by image processing software; the operation of defogging the sample can be completed through an end-to-end neural network model and can also be realized according to an atmospheric degradation model; algorithms for making the image clear can be wiener filtering, fourier transform, etc.; the method for increasing the pixels of the clear image of the sample is the super-resolution of the image, and the method for the super-resolution of the image comprises an interpolation method, a sparse representation (dictionary learning) based method and the like. In the image restoration process, the image is colored to obtain a sample color image. And then carrying out color balance on the sample color image to obtain a sample color balance image. This step is for the purpose of making the color transition in the image more natural. Then, the step of carrying out defogging operation on the sample color balance image to obtain a sample defogged image enables the foggy image to be clearer than the fogless image, and the operation of the step does not influence the image quality. After defogging, the definition of the defogged image of the sample needs to be adjusted to obtain a clear image of the sample, so that the image quality is better. And finally, increasing pixels of the clear sample image to obtain a sample target image corresponding to the sample image, wherein the step is the super-resolution operation of the image and is the supplement of the pixels of the image details.

Step 302, inputting the first image into the target detection model to obtain at least one target area of the first image.

In some embodiments, the target detection model is used to identify at least one target object image in the first image and set a corresponding target region for each of the at least one target object image. As an example, the target detection model may enable determination of the target area through a target detection network. As an example, the target detection network may be an R-CNN network, SPPNet network, Fast R-CNN network, or the like.

Step 303, inputting the target area into the area processing model for the target area in the at least one target area to obtain a target modification area corresponding to the target area.

In some embodiments, the region processing model is used to repair image features of the target object image within the target region. The target modification area is obtained by further processing the detail of the target area in the image. As an example, the region processing model may be an image inpainting model based on convolutional self-encoding, an image inpainting model based on generation of a countermeasure network, an image inpainting model based on a recurrent neural network.

Step 304, constructing a second image based on at least one target modification area corresponding to the at least one target area.

In some embodiments, the second image is an image of the target area with further detail restoration based on the first image.

And 305, performing image enhancement on the second image to obtain a target image.

In some embodiments, specific implementations of

steps

304 and 305 and technical effects thereof may refer to

steps

204 and 205 in the embodiment corresponding to fig. 2, and are not described herein again.

As can be seen from fig. 3, compared with the description of some embodiments corresponding to fig. 2, the flow 300 of the method for repairing an image in some embodiments corresponding to fig. 3 embodies the operations of how to obtain the first image, how to determine the target area, and how to modify the target area. By preprocessing the model to the first image. The image characteristics can be controlled integrally. In addition, the order of image inpainting in the pre-model is also important, and different operation orders may bring different results. The operation steps of repairing the image can be reasonably distributed by training the pre-model. And obtaining a target area through a target detection model. By using the target detection model, the area needing to be specially repaired can be well found, so that the details of the repaired image can be more fully embodied. And obtaining a target modification area through the area processing model. And image restoration is carried out aiming at the area, so that the restoration is more targeted and the restoration effect is better.

With further reference to fig. 4, a flow 400 of further embodiments of a method of inpainting an image is illustrated. The flow 400 of the method for repairing an image comprises the following steps:

step 401, preprocessing an image to be restored to obtain a first image.

At step 402, at least one target region in a first image is determined.

And step 403, adjusting the definition of the target area to obtain a target clear area.

In some embodiments, adjusting the sharpness of the target region may be performed by non-neighborhood filtering, and least squares filtering.

And step 404, increasing pixels of the target clear area to obtain a target modification area.

In some embodiments, increasing the pixels of the target region may be performed by a method Based on local Embedding (Neighbor Embedding), an instance-Based (Example-Based) hyper-resolution reconstruction method, or the like, as examples.

Step 405, constructing a second image based on at least one target modification area corresponding to at least one target area.

And 406, performing image enhancement on the second image to obtain a target image.

In some embodiments, specific implementations of

steps

401, 402, 405, and 406 and technical effects thereof may refer to

steps

201, 202, 204, and 205 in the embodiment corresponding to fig. 2, and are not described herein again.

As can be seen from fig. 4, compared with the description of some embodiments corresponding to fig. 3, the flow 400 of the method for repairing an image in some embodiments corresponding to fig. 4 represents one implementation method for repairing a target area. The detail adjustment of the target area is done by adjusting the sharpness of the target area and increasing the pixels of the target area. Therefore, the image details can be well supplemented, and the problem that local details of the image are possibly lost after the whole image is repaired is solved.

With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an apparatus for repairing an image, which correspond to those of the method embodiments shown in fig. 2, and which may be applied in particular in various electronic devices.

As shown in fig. 5, an apparatus 500 for restoring an image of some embodiments includes: the image restoration method includes a preprocessing unit 501 configured to preprocess an image to be restored to obtain a first image, where the image to be restored is an image with missing or unobvious image features; a determining unit 502 configured to determine at least one target region in the first image; a processing unit 503, configured to process a target area of the at least one target area to obtain a target modification area; a construction unit 504 configured to construct a second image based on at least one target modification region corresponding to the at least one target region; and an enhancing unit 505 configured to perform image enhancement on the second image to obtain a target image.

In an optional implementation of some embodiments, the pre-processing unit 501 is further configured to: and inputting the image to be repaired into the preprocessing model to obtain a first image.

In an alternative implementation of some embodiments, the sample target image is obtained by: identifying a sample target object within a sample image; adding colors based on the sample target object to obtain a sample color image; carrying out color balance processing on the sample color image to obtain a sample color balance image; carrying out defogging treatment on the sample color balance image to obtain a sample defogged image; adjusting the definition of the defogged image of the sample to obtain a clear image of the sample; and increasing pixels of the clear sample image to obtain a sample target image corresponding to the sample image.

In an optional implementation of some embodiments, the determining unit 502 is further configured to: the first image is input into a target detection model to obtain at least one target area of the first image, and the target detection model is used for identifying at least one target object image in the first image and setting a corresponding target area for each target object image in the at least one target object image.

In an optional implementation of some embodiments, the processing unit 503 is further configured to: and inputting the target area into an area processing model for the target area in the at least one target area to obtain a target modification area corresponding to the target area, wherein the area processing model is used for repairing the image characteristics of the target object image in the target area.

In an optional implementation of some embodiments, the processing unit 503 is further configured to: adjusting the definition of the target area to obtain a target clear area; and increasing the pixels of the target clear area to obtain a target modification area.

It will be appreciated that the storage elements described in the apparatus 500 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting advantages described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and are not described herein again.

Referring now to fig. 6, a schematic diagram of an electronic device (e.g., the server or terminal device of fig. 1) 600 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic device in some embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle-mounted terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.

It should be noted that the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: preprocessing an image to be restored to obtain a first image, wherein the image to be restored is an image with missing or unobvious image features; determining at least one target region in the first image; processing the target area of at least one target area to obtain a target modification area; constructing a second image based on at least one target modification area corresponding to at least one target area; and performing image enhancement on the second image to obtain a target image.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes a preprocessing unit, a determination unit, a processing unit, a construction unit, and an enhancement unit. The names of the units do not in some cases constitute a limitation on the units themselves, and for example, a preprocessing unit may also be described as a "unit that preprocesses an image to be repaired".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

According to one or more embodiments of the present disclosure, there is provided a method of repairing an image, including: preprocessing an image to be restored to obtain a first image, wherein the image to be restored is an image with missing or unobvious image features; determining at least one target region in the first image; processing the target area of at least one target area to obtain a target modification area; constructing a second image based on at least one target modification area corresponding to at least one target area; and performing image enhancement on the second image to obtain a target image.

According to one or more embodiments of the present disclosure, preprocessing an image to be restored to obtain a first image includes: and inputting the image to be repaired into the preprocessing model to obtain a first image.

According to one or more embodiments of the present disclosure, the preprocessing model is obtained by: the method comprises the steps of obtaining a plurality of sample images and a sample target image corresponding to each sample image in the plurality of sample images, wherein the sample images are images with missing or unobvious image features, and the sample target images are images with complete image features corresponding to the sample images; and taking each sample image in the plurality of sample images as input, taking a sample target image corresponding to each sample image in the plurality of sample images as output, and training to obtain the preprocessing model.

According to one or more embodiments of the present disclosure, a sample target image is obtained by: identifying a sample target object within a sample image; adding colors based on the sample target object to obtain a sample color image; carrying out color balance processing on the sample color image to obtain a sample color balance image; carrying out defogging treatment on the sample color balance image to obtain a sample defogged image; adjusting the definition of the defogged image of the sample to obtain a clear image of the sample; and increasing pixels of the clear sample image to obtain a sample target image corresponding to the sample image.

According to one or more embodiments of the present disclosure, determining at least one target region in a first image comprises: the first image is input into a target detection model to obtain at least one target area of the first image, and the target detection model is used for identifying at least one target object image in the first image and setting a corresponding target area for each target object image in the at least one target object image.

According to one or more embodiments of the present disclosure, for a target area of at least one target area, processing the target area to obtain a target modification area includes: and inputting the target area into an area processing model for the target area in at least one target area to obtain a target modification area corresponding to the target area, wherein the area processing model is used for repairing the image characteristics of the target object image in the target area.

According to one or more embodiments of the present disclosure, inputting a target area into a region processing model, and obtaining a target modification region corresponding to the target area, includes: adjusting the definition of the target area to obtain a target clear area; and increasing the pixels of the target clear area to obtain a target modification area.

According to one or more embodiments of the present disclosure, there is provided an apparatus for repairing an image, including: the image restoration device comprises a preprocessing unit, a storage unit and a processing unit, wherein the preprocessing unit is configured to preprocess an image to be restored to obtain a first image, and the image to be restored is an image with missing or unobvious image features; a determination unit configured to determine at least one target region in the first image; the processing unit is configured to process a target area of at least one target area to obtain a target modification area; a construction unit configured to construct a second image based on at least one target modification region corresponding to the at least one target region; and the enhancement unit is configured to perform image enhancement on the second image to obtain a target image.

In accordance with one or more embodiments of the present disclosure, the pre-processing unit is further configured to: and inputting the image to be repaired into the preprocessing model to obtain a first image.

According to one or more embodiments of the present disclosure, the determining unit is further configured to: the first image is input into a target detection model to obtain at least one target area of the first image, and the target detection model is used for identifying at least one target object image in the first image and setting a corresponding target area for each target object image in the at least one target object image.

In accordance with one or more embodiments of the present disclosure, the processing unit is further configured to: and inputting the target area into an area processing model for the target area in the at least one target area to obtain a target modification area corresponding to the target area, wherein the area processing model is used for repairing the image characteristics of the target object image in the target area.

In accordance with one or more embodiments of the present disclosure, the processing unit is further configured to: adjusting the definition of the target area to obtain a target clear area; and increasing the pixels of the target clear area to obtain a target modification area.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims

1. A method of repairing an image, comprising:

preprocessing an image to be restored to obtain a first image, wherein the image to be restored is an image with missing or unobvious image features;

determining at least one target region in the first image;

processing the target area of the at least one target area to obtain a target modification area;

constructing a second image based on at least one target modification area corresponding to the at least one target area;

and performing image enhancement on the second image to obtain a target image.

2. The method according to claim 1, wherein the preprocessing the image to be repaired to obtain the first image comprises:

and inputting the image to be repaired into a preprocessing model to obtain a first image.

3. The method of claim 2, wherein the pre-processing model is derived by:

the method comprises the steps of obtaining a plurality of sample images and a sample target image corresponding to each sample image in the plurality of sample images, wherein the sample images are images with missing or unobvious image features, and the sample target images are images with complete image features corresponding to the sample images;

and taking each sample image in the plurality of sample images as input, taking a sample target image corresponding to each sample image in the plurality of sample images as output, and training to obtain the preprocessing model.

4. The method of claim 3, wherein the sample target image is obtained by:

identifying a sample target object within the sample image;

adding colors based on the sample target object to obtain a sample color image;

carrying out color balance processing on the sample color image to obtain a sample color balance image;

carrying out defogging treatment on the sample color balance image to obtain a sample defogged image;

adjusting the definition of the defogged image of the sample to obtain a clear image of the sample;

and increasing the pixels of the sample sharp image to obtain a sample target image corresponding to the sample image.

5. The method of claim 1, wherein the determining at least one target region in the first image comprises:

inputting the first image into a target detection model to obtain at least one target area of the first image, wherein the target detection model is used for identifying at least one target object image in the first image and setting a corresponding target area for each target object image in the at least one target object image.

6. The method of claim 5, wherein the processing the target area of the at least one target area to obtain a target modified area comprises:

and inputting the target area into an area processing model for the target area in the at least one target area to obtain a target modification area corresponding to the target area, wherein the area processing model is used for repairing the image characteristics of the target object image in the target area.

7. The method of claim 6, wherein the inputting the target region into a region processing model, resulting in a target modified region corresponding to the target region, comprises:

adjusting the definition of the target area to obtain a target clear area;

and increasing the pixels of the target clear area to obtain the target modification area.

8. An apparatus for acquiring an image, comprising:

the image restoration method comprises the steps that a preprocessing unit is configured to preprocess an image to be restored to obtain a first image, wherein the image to be restored is an image with missing or unobvious image features;

a determination unit configured to determine at least one target region in the first image;

the processing unit is configured to process a target area of the at least one target area to obtain a target modification area;

a construction unit configured to construct a second image based on at least one target modification region corresponding to the at least one target region;

and the enhancing unit is configured to enhance the second image to obtain a target image.

9. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1 to 7.