CN116228605B - Image complement method, device, computer equipment and storage medium - Google Patents
Image complement method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116228605B CN116228605B CN202310513801.7A CN202310513801A CN116228605B CN 116228605 B CN116228605 B CN 116228605B CN 202310513801 A CN202310513801 A CN 202310513801A CN 116228605 B CN116228605 B CN 116228605B
- Authority
- CN
- China
- Prior art keywords
- image
- missing
- sub
- region
- noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000295 complement effect Effects 0.000 title claims abstract description 129
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000009467 reduction Effects 0.000 claims abstract description 51
- 238000012545 processing Methods 0.000 claims abstract description 42
- 238000000605 extraction Methods 0.000 claims abstract description 27
- 238000000638 solvent extraction Methods 0.000 claims abstract description 18
- 238000010586 diagram Methods 0.000 claims abstract description 13
- 238000004590 computer program Methods 0.000 claims description 20
- 230000000750 progressive effect Effects 0.000 claims description 17
- 230000002829 reductive effect Effects 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 11
- 238000012217 deletion Methods 0.000 claims description 10
- 230000037430 deletion Effects 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 238000012937 correction Methods 0.000 claims 2
- 230000008569 process Effects 0.000 description 9
- 238000009792 diffusion process Methods 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 7
- 230000000149 penetrating effect Effects 0.000 description 5
- 238000005315 distribution function Methods 0.000 description 4
- 238000012952 Resampling Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The application relates to an image complement method, an image complement device, computer equipment and a storage medium. The method comprises the following steps: performing linear extraction processing on the first image to be complemented to obtain linear information; the first image comprises a missing area for missing image content; partitioning the missing region based on the linear information to obtain a plurality of sub-missing regions; the first image is subjected to noise adding for multiple times according to the number of the sub-missing areas, and Gaussian noise diagrams which accord with the number and are different in Gaussian noise are obtained; selecting target Gaussian noise maps from the Gaussian noise maps conforming to the number of the target Gaussian noise maps for each sub-missing region, and extracting image content corresponding to the sub-missing region from the target Gaussian noise maps to obtain a noise map of the sub-missing region; combining the noise image of each sub-missing region with the first image to obtain a preliminary complement image; and carrying out noise reduction treatment on the preliminary complement image to obtain a target image. The complement image content generated by the method is more accurate.
Description
Technical Field
The present application relates to the field of image technologies, and in particular, to an image complement method, an image complement device, a computer device, and a storage medium.
Background
For various reasons, there are cases where the image content is missing in a partial region in the image, thereby affecting the visual effect of the image. In the traditional method, the idea of block copy is proposed, and the missing image content in the image is complemented. Specifically, around the missing region of the missing image content, an image block similar to the missing image content is quickly searched according to the nearest neighbor principle, and the image content of the searched image block is copied to the missing region.
However, the image content of the missing region complemented by the conventional method is not accurate enough and has a larger difference from the image content actually lost by the missing region, so that the complemented image has larger visual deviation.
Disclosure of Invention
In view of the foregoing, it is desirable to provide an image complement method, apparatus, computer device, computer-readable storage medium, and computer program product that can obtain accurate complement of image content.
In a first aspect, the present application provides an image complement method, including:
performing linear extraction processing on the first image to be complemented to obtain linear information; the first image comprises a missing area for missing image content;
Partitioning the missing region based on the linear information to obtain a plurality of sub-missing regions;
carrying out noise adding on the first image for multiple times according to the number of the sub-missing areas to obtain Gaussian noise diagrams conforming to the number; the gaussian noise in each gaussian noise plot is different;
selecting target Gaussian noise maps from the Gaussian noise maps conforming to the number of the target Gaussian noise maps for each sub-missing region, and extracting image content corresponding to the sub-missing region from the target Gaussian noise maps to obtain a noise map of the sub-missing region;
combining the noise image of each sub-missing region with the first image to obtain a preliminary complement image;
and carrying out noise reduction treatment on the preliminary complement image to obtain a target image.
In some embodiments, the first image includes a known region having image content therein; the straight line information includes a global straight line; the global straight line is a straight line that penetrates through the missing region in the first image; the method for extracting the straight line of the first image to be complemented to obtain the straight line information comprises the following steps:
performing linear extraction processing on a known region in the first image to be complemented to obtain a local linear in the known region;
clustering the local straight lines, and fusing the local straight lines gathered into one type to obtain a plurality of candidate straight lines;
A global straight line is selected from the plurality of candidate straight lines.
In some embodiments, the blocking of the missing regions based on the line information results in a plurality of sub-missing regions, including:
forming a binary image based on the extracted global straight line to obtain a straight line binary image; the linear region pixel value in the linear binary image is different from the non-linear region pixel value;
and performing phase operation on the linear binary image and the missing region to block the missing region to obtain a plurality of sub-missing regions.
In some embodiments, performing an anding operation on the rectilinear binary image and the missing regions to block the missing regions to obtain a plurality of sub-missing regions, comprising:
extracting edge features of the first image under the condition that the missing area is at the image edge of the first image;
performing phase-separating operation on the linear binary image and the missing region to divide the missing region so as to obtain a non-closed missing sub-region positioned at the edge of the image and a closed missing sub-region positioned at the edge of the non-image;
and adding corresponding image edges to the non-closed deletion sub-regions based on the edge features to obtain closed deletion sub-regions positioned at the image edges.
In some embodiments, the method further comprises:
Determining an initial mask of the first image;
performing inverse operation on the initial mask to obtain a mask of the missing region;
partitioning the mask of the missing region based on the linear information to obtain masks of all sub-missing regions;
extracting image content corresponding to the sub-missing region from the target Gaussian noise map to obtain a noise map of the sub-missing region, wherein the method comprises the following steps:
and extracting image content corresponding to the sub-missing region from the target Gaussian noise map according to the mask of the sub-missing region to obtain a noise map of the sub-missing region.
In some embodiments, performing noise reduction processing on the preliminary complement image to obtain a target image includes:
taking the preliminary full image as a current full image, and carrying out noise reduction treatment on the current full image to obtain a progressive full image;
and taking the progressive complement image as a current complement image, returning to execute the step of carrying out noise reduction treatment on the current complement image to obtain the progressive complement image so as to carry out iterative noise reduction until iteration is stopped, and taking the progressive complement image after iteration is stopped as a target image.
In some embodiments, the target image is obtained by denoising the preliminary complement image through a trained image generation model; the trained image generation model is obtained by training a target building image set which is in different scenes and contains different building textures; the target building image in the target building image set is obtained by preprocessing the original building image; the step of preprocessing the original building image comprises:
Projecting and correcting the original building image to obtain a corrected image of the original building image;
clipping the projection alignment image to obtain a clipping image meeting the preset resolution condition;
and determining a target clipping image meeting the building texture semantics from the clipping images, and taking the target clipping image as a target building image.
In a second aspect, the present application also provides an image complement apparatus, including:
the linear extraction module is used for carrying out linear extraction processing on the first image to be complemented to obtain linear information; the first image comprises a missing area for missing image content; partitioning the missing region based on the linear information to obtain a plurality of sub-missing regions;
the noise adding module is used for adding noise to the first image for a plurality of times according to the number of the sub-missing areas to obtain Gaussian noise diagrams conforming to the number; the gaussian noise in each gaussian noise plot is different;
the merging module is used for selecting a target Gaussian noise map from the Gaussian noise maps conforming to the number of the target Gaussian noise maps for each sub-missing region, extracting image content corresponding to the sub-missing region from the target Gaussian noise map and obtaining a noise map of the sub-missing region; combining the noise image of each sub-missing region with the first image to obtain a preliminary complement image;
And the noise reduction module is used for carrying out noise reduction processing on the preliminary complement image to obtain a target image.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the method described above when the processor executes the computer program.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the method described above.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, implements the steps of the method described above.
According to the image complement method, the device, the computer equipment and the storage medium, the linear information extracted from the first image to be complemented is used for partitioning the missing area of the first image, and different Gaussian noises are added to the sub-missing areas obtained by partitioning, so that Gaussian noise images conforming to the number of the sub-missing areas are obtained, target Gaussian noise images are selected from the Gaussian noise images conforming to the number, image contents corresponding to the sub-missing areas are extracted from the target Gaussian noise images, the noise images of the sub-missing areas are obtained, the noise images of the sub-missing areas and the first image are combined to obtain a preliminary complement image, noise reduction processing is carried out on the preliminary complement image to obtain a target image, and as the linear information is used for partitioning the missing area, different Gaussian noises are added to the different sub-missing areas, in the process of obtaining the target image by carrying out noise reduction processing on the complete image, the complete image contents matched with the sizes of the sub-missing areas are generated, the complete image contents of the sub-missing areas are prevented from being matched with the actual image contents of the sub-missing areas, and the complete image has a better visual effect of the complete image is obtained, and the complete image with the actual image is more accurate, and the difference between the complete image is obtained.
Drawings
FIG. 1 is a flow chart of an image complement method in one embodiment;
FIG. 2 is a flow chart of an image complement method according to another embodiment;
FIG. 3 is a flow chart of an image complement method according to another embodiment;
FIG. 4 is a block diagram of an image complement apparatus in one embodiment;
FIG. 5 is a block diagram of a straight line extraction module in one embodiment;
fig. 6 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It will be appreciated that in many application scenarios, an image with complete image content is required to meet a specific requirement, however, for various reasons, there is a situation that the image content is missing in a partial area of the image, and thus the specific requirement cannot be met. For example, when a building is three-dimensionally reconstructed, building textures in images obtained by photographing the building from different perspectives need to be mapped onto a three-dimensional building model to obtain the three-dimensional building model with the building textures, however, because the building is photographed from different perspectives and is affected by shielding, illumination or position, building texture missing often exists in the photographed images, and the building textures of the three-dimensional building model are incomplete. Therefore, image complement is required for the image in which the image content is missing.
In one embodiment, as shown in fig. 1, there is provided an image complement method, which is exemplified as the method applied to a computer device, and includes the following steps:
step 101, performing linear extraction processing on a first image to be complemented to obtain linear information; the first image includes a missing region in which the image content is missing.
The first image to be complemented comprises a missing area missing the image content and a known area with the image content.
The straight line information is a characteristic straight line in the first image to be complemented, and the characteristic straight line is divided into a global straight line and a local straight line according to the length of the characteristic straight line, namely the straight line information comprises the global straight line and the local straight line.
The global straight line is a straight line that intersects the missing region in the first image. The penetration includes traversing, or beveling, and thus the global straight line refers to any one of a straight line traversing a missing region in the first image, or a straight line beveling a missing region in the first image.
The computer device performs linear extraction processing on a known region in the first image to be complemented to obtain linear information in the first image to be complemented.
In some embodiments, the step of performing a linear extraction process on the first image to be complemented to obtain linear information includes: and calculating a gradient field of the first image to be complemented, and determining straight line information according to the gradient field.
And 102, partitioning the missing region based on the linear information to obtain a plurality of sub-missing regions.
Illustratively, the computer device blocks the missing region based on a global straight line to obtain a plurality of sub-missing regions, i.e. the missing region is divided by a straight line passing through the missing region in the first image to obtain a plurality of sub-missing regions.
In some embodiments, in a case where the missing region is at an image edge of the first image, the missing region is segmented based on a global straight line and edge features of the first image, resulting in a plurality of sub-missing regions.
Step 103, carrying out noise adding on the first image for multiple times according to the number of the sub-missing areas to obtain Gaussian noise diagrams conforming to the number; the gaussian noise in each gaussian noise plot is different.
Illustratively, the computer device determines the number of sub-missing regions and performs noise adding on the first image multiple times according to the number of sub-missing regions, that is, adds different gaussian noise to the first image on the basis of the first image, and generates a gaussian noise map each time the first image is subjected to noise adding. Therefore, the number of times of noise adding is carried out on the first image to obtain Gaussian noise images conforming to the number, namely the number of Gaussian noise images is consistent with the number of sub-missing areas.
To facilitate understanding, a process of denoising the first image multiple times by the number of sub-missing regions to obtain a gaussian noise map of a consistent number is illustrated. For example, if the number of the sub-missing regions is 3, adding 3 different gaussian noise a, gaussian noise B and gaussian noise C to the first image respectively, so as to obtain 3 gaussian noise patterns, that is, adding the gaussian noise a to the first image when the first noise is added; adding Gaussian noise B to the first image during the second noise adding; at the time of the third noise addition, gaussian noise B is added to the first image. It will be appreciated that one gaussian noise is selected from the 3 gaussian noise types each time the first image is noisy, and that the gaussian noise is different from one gaussian noise to another. In addition, the order of adding the gaussian noise may be preset, for example, the gaussian noise a, the gaussian noise B, and the gaussian noise C may be sequentially added, or may be random, which is not limited herein.
And 104, selecting a target Gaussian noise map from the Gaussian noise maps conforming to the number of the target Gaussian noise maps for each sub-missing region, and extracting image content corresponding to the sub-missing region from the target Gaussian noise map to obtain the noise map of the sub-missing region.
The target Gaussian noise map is a Gaussian noise map for providing image content corresponding to the sub-missing region. It will be appreciated that since gaussian noise in the extracted target gaussian noise map is taken as the image content corresponding to the sub-absent area, it is necessary to determine the target gaussian noise map corresponding to the sub-absent area from the coincident number of gaussian noise maps.
For each sub-missing region, the computer device selects a target Gaussian noise map which can provide image content corresponding to the sub-missing region from the Gaussian noise maps with the consistent quantity, and extracts the image content corresponding to the sub-missing region from the target Gaussian noise map according to a mask of the sub-missing region to obtain a noise map of the sub-missing region.
In some embodiments, for each sub-missing region, the step of selecting the target gaussian noise map from the consistent number of gaussian noise maps comprises: and randomly selecting a target Gaussian noise map from the Gaussian noise maps conforming to the number for each sub-missing region.
In some embodiments, for each sub-missing region, the step of selecting the target gaussian noise map from the consistent number of gaussian noise maps comprises: and selecting a target Gaussian noise map from the sequences of adding Gaussian noise in the Gaussian noise maps conforming to the number for each sub-missing region. That is, the obtained gaussian noise map is sequentially divided into the respective sub-absent areas in the order of adding gaussian noise.
And 105, merging the noise image of each sub-missing region with the first image to obtain a preliminary complement image.
Illustratively, the computer device combines the noise map of each sub-missing region with the first image, i.e., uses the noise map of each sub-missing region as the image content of the missing region in the first image, so as to perform preliminary complement on the first image, and obtain a preliminary complement image.
In some embodiments, each gaussian noise map is denoised after the step of deriving a consistent number of gaussian noise maps. The first image is denoised before combining the noise figure for each sub-absent area with the first image. Therefore, when the noise map of each sub-missing region is combined with the first image, the noise map of each sub-missing region after noise reduction is combined with the first image after noise addition. It can be understood that the preliminary complement image is an image obtained when the first image to be complemented is first complemented, and therefore, an image obtained by combining the noise image of each sub-missing region and the first image, or an image obtained by combining the noise image of each sub-missing region after noise reduction and the first image after noise addition may be referred to as a preliminary complement image.
It can be understood that by denoising each gaussian noise figure and adding noise to the first image, the difference in image content between the missing region and the known region of the first image can be reduced, so that when the preliminary complement image is subjected to the denoising process, the time required for the denoising process can be reduced, and the speed of obtaining the target image can be increased.
And 106, carrying out noise reduction treatment on the preliminary complement image to obtain a target image.
The target image is an image in which the image content in the missing region is completed and the image content in the missing region has semantic consistency and geometric consistency with the image content of the known region.
It is understood that semantic consistency refers to consistency of the completed image content with the attributes of the image content of the known region, that is, that the completed image content and the image content of the known region belong to the same target object. For example, the image content of the known region is a building texture, and the complement image content is a lawn texture, indicating that the complement image content does not have semantic consistency with the image content of the known region; for another example, the image content of the known region is a building texture and the completed image content is a building texture, indicating that the completed image content has semantic consistency with the image content of the known region. Geometric consistency means that the image content of the complement remains geometrically consistent with the image content of the known region. When the completed image content has semantic consistency and geometric consistency with the image content of the known area, the completed image content is accurate.
Illustratively, the computer device performs noise reduction processing on the preliminary complement image to remove noise in the preliminary complement image, and obtains a target image.
In some embodiments, the preliminary complement image is resampled before the noise reduction processing is performed on the preliminary complement image, so as to obtain a resampled preliminary complement image, and the noise reduction processing is performed on the resampled preliminary complement image, so as to obtain the target image. It can be understood that resampling is performed before the noise reduction processing is performed on the preliminary complement image, so that the time required for the noise reduction processing can be reduced, and the speed of obtaining the target image can be increased.
In the above embodiment, since the missing regions are segmented by using the straight line information, and different gaussian noise is added for different sub-missing regions, so as to obtain the noise map of different sub-missing regions, in the process of performing noise reduction processing on the preliminary complement image to obtain the target image, the complement image content matched with the size of each sub-missing region is generated for each sub-missing region, so that the phenomenon of fitting is avoided, the generated complement image content is more accurate, the difference between the complement image content and the image content actually lost by the missing region is reduced, and the obtained target image has better visual effect.
In some embodiments, the first image includes a known region having image content therein; the straight line information includes a global straight line; the global straight line is a straight line that penetrates through the missing region in the first image; the method comprises the steps of performing linear extraction processing on a first image to be complemented to obtain linear information, and comprises the following steps: performing linear extraction processing on a known region in the first image to be complemented to obtain a local linear in the known region; clustering the local straight lines, and fusing the local straight lines gathered into one type to obtain a plurality of candidate straight lines; a global straight line is selected from the plurality of candidate straight lines.
Wherein the local straight line is a characteristic straight line in the known region. The length of the local straight line is shorter than the global straight line.
The candidate straight line is a straight line that intersects the first image. It will be appreciated that the first image includes a missing region of missing image content and a known region of image content, and thus the candidate straight lines include a straight line that intersects the missing region in the first image and a straight line that intersects the known region in the first image.
The computer equipment performs linear extraction processing on the known region in the first image to be complemented to obtain a local linear of the known region in the first image; fusing the local straight lines gathered into one type to obtain a plurality of candidate straight lines, namely a straight line penetrating through a missing region in the first image and a straight line penetrating through a known region in the first image; and selecting a global straight line from the plurality of candidate straight lines, namely selecting a straight line penetrating through the missing area in the first image from the plurality of candidate straight lines.
In the above embodiment, the local straight lines of the known region in the first image to be complemented are extracted, and clustered and fused to obtain a plurality of candidate straight lines, and the global straight line penetrating through the missing region of the first image is determined from the plurality of candidate straight lines, so that the missing region is segmented through the global straight line.
In some embodiments, the step of partitioning the missing region based on the line information to obtain a plurality of sub-missing regions includes: forming a binary image based on the extracted global straight line to obtain a straight line binary image; the linear region pixel value in the linear binary image is different from the non-linear region pixel value; and performing phase operation on the linear binary image and the missing region to block the missing region to obtain a plurality of sub-missing regions.
Illustratively, the computer device forms a binary image based on the extracted global straight line, and obtains a straight line binary image with a straight line region pixel value different from a non-straight line region pixel value, that is, the pixel value of the non-straight line region is 0, and the pixel value of the straight line region is 1; or the pixel value of the non-linear region is 1 and the pixel value of the linear region is 0. And performing phase-inversion operation on the linear binary image and the missing region to block the missing region to obtain a plurality of sub-missing regions, namely after performing phase-inversion operation, each region corresponding to the non-linear region is a sub-missing region.
In the implementation, the linear binary image of the global straight line and the missing region are subjected to the phase operation so as to block the missing region to obtain a plurality of sub-missing regions, so that in the process of carrying out noise reduction processing on the preliminary complement image to obtain the target image, the complement image content matched with the size of each sub-missing region is generated for each sub-missing region, the phenomenon of fitting is avoided, and the generated complement image content is more accurate.
In some embodiments, the step of performing an anding operation on the rectilinear binary image and the missing regions to block the missing regions to obtain a plurality of sub-missing regions includes: extracting edge features of the first image under the condition that the missing area is at the image edge of the first image; performing phase-separating operation on the linear binary image and the missing region to divide the missing region so as to obtain a non-closed missing sub-region positioned at the edge of the image and a closed missing sub-region positioned at the edge of the non-image; and adding corresponding image edges to the non-closed deletion sub-regions based on the edge features to obtain closed deletion sub-regions positioned at the image edges.
The edge feature refers to the edge feature of the missing region when the missing region is at the image edge of the first image.
It will be appreciated that, since the straight line extraction can only extract straight lines of known regions located in the first image, in the case where the missing region is located at an image edge of the first image, i.e. the image edge of the first image lacks image content, dividing the missing region results in a non-closed missing sub-region located at the image edge and a closed missing sub-region located at the non-image edge. It is thus necessary to add a corresponding image edge to the non-occluded missing sub-region based on the edge features, resulting in an occluded missing sub-region at the image edge. That is, each missing sub-region includes a closed missing sub-region at a non-image edge and a closed missing sub-region at an image edge.
In the above embodiment, when the missing region is located at the image edge of the first image, a corresponding image edge is added to the non-closed missing sub-region based on the edge feature, so as to obtain the closed missing sub-region located at the image edge, so that each obtained missing sub-region is closed, and in the process of performing noise reduction processing on the preliminary complement image to obtain the target image, the complement image content matched with the size of each sub-missing region is generated for each sub-missing region, and the phenomenon of overfitting is avoided, so that the generated complement image content is more accurate. In some embodiments, the method further comprises: determining an initial mask of the first image; performing inverse operation on the initial mask to obtain a mask of the missing region; partitioning the mask of the missing region based on the linear information to obtain masks of all sub-missing regions; extracting image content corresponding to the sub-missing region from the target Gaussian noise map to obtain a noise map of the sub-missing region, wherein the step comprises the following steps: and extracting image content corresponding to the sub-missing region from the target Gaussian noise map according to the mask of the sub-missing region to obtain a noise map of the sub-missing region.
Wherein the initial mask is a mask of the first image. It is understood that the initial mask refers to a binary image in which pixels in a known region of the first image are white and pixels in a missing region are black. Therefore, the original mask is inverted, that is, pixels in a known region of the first image are changed from white to black, and pixels in a missing region of the first image are changed from black to white, thereby obtaining a mask of the missing region. That is, the mask of the missing region refers to a binary image in which pixels in a known region of the first image are black and pixels in the missing region are white.
Illustratively, the computer device determines an initial mask of the first image, and performs a reversal operation on the initial mask to obtain a mask of the missing region; partitioning the mask of the missing region based on a global straight line to obtain masks of all sub-missing regions; and extracting image content corresponding to the sub-missing region from the target Gaussian noise map according to the mask of the sub-missing region, namely multiplying the mask of the sub-missing region by the target Gaussian noise map, extracting Gaussian noise in the target Gaussian noise map, and taking the Gaussian noise in the extracted target Gaussian noise map as the image content corresponding to the sub-missing region to obtain the noise map of the sub-missing region.
In the above embodiment, the mask of the missing region is obtained by performing the inverse operation on the initial mask of the first image, the mask of the missing region is segmented based on the global line, the mask of the sub-missing region is obtained, and the image content corresponding to the sub-missing region is extracted from the target gaussian noise map according to the mask of the sub-missing region, so as to obtain the noise map of the sub-missing region, so that the first image to be complemented is primarily complemented through the noise map of the sub-missing region, and the primary complement image is obtained.
In some embodiments, performing noise reduction processing on the preliminary complement image to obtain a target image includes: taking the preliminary full image as a current full image, and carrying out noise reduction treatment on the current full image to obtain a progressive full image; and taking the progressive complement image as a current complement image, returning to execute the step of carrying out noise reduction treatment on the current complement image to obtain the progressive complement image so as to carry out iterative noise reduction until iteration is stopped, and taking the progressive complement image after iteration is stopped as a target image.
In the above embodiment, the iterative noise reduction processing is performed on the preliminary complement image, so that the generated complement image content is more accurate, and the difference between the complement image content and the image content actually lost in the missing area is reduced, so that the obtained target image has a better visual effect.
In some embodiments, the target image is obtained by denoising the preliminary complement image through a trained image generation model; the trained image generation model is obtained by training a target building image set which is in different scenes and contains different building textures; the target building image in the target building image set is obtained by preprocessing the original building image; the step of preprocessing the original building image comprises: performing projection alignment on the original building image to obtain a projection alignment image of the original building image; clipping the projection alignment image to obtain a clipping image meeting the preset resolution condition; and determining a target clipping image meeting the building texture semantics from the clipping images, and taking the target clipping image as a target building image.
The trained image generation model performs noise reduction processing on the preliminary complement image to obtain a model of the target image, namely the trained image generation model performs noise reduction processing on the preliminary complement image to complement missing image content, so that the complementary image content has semantic consistency and geometric consistency with the image content of a known area.
It will be appreciated that training the untrained image generation model with the training sample image is required before the preliminary complement image is noise reduced using the image generation model. Depending on the application scenario in which the trained image generation model is used, different training sample images are required to train the untrained image generation model.
For example, when the application scene is the image content of the complement building image, the training sample image is a set of target building images under different scenes and including different building textures. The target building image is obtained by preprocessing an original building image, so that the original building is an image which is shot from different scenes and contains different building textures. The different scenes refer to a plurality of shooting scenes of the original building image, including scenes such as schools, apartments or enterprises, namely, the original building image is obtained by shooting buildings in the scenes such as schools, apartments or enterprises. In order to ensure the effectiveness of training, the original building image needs to be preprocessed to obtain the target building image.
It can be understood that, because the problem of oblique angle shooting may exist when the original building image is shot, the obtained building image in the original building image is oblique, so the computer device performs projection alignment on the original building image, that is, performs projection alignment processing on the building image in the original building image, and obtains a projection alignment image. In order to reduce the loss of the resolution of the projection alignment image, the computer equipment performs clipping processing on the projection alignment image to obtain a clipping image meeting the preset resolution condition, and deletes the blurred clipping image in the clipping image. Next, the computer device determines a target clipping image satisfying the building texture semantics from the clipping images, that is, performs semantic segmentation on the clipping images, determines a target clipping image containing a large amount of building textures in the clipping images, and takes the target clipping image as a target building image. That is, the target building image is obtained by performing a series of preprocessing such as projection alignment, clipping, and semantic segmentation on the original building image.
In the above embodiment, since the target building image in the target building image set is obtained by performing a series of preprocessing including projection alignment, clipping and semantic segmentation on the original building image, the characteristics of building textures are retained in the target building image, and the completed image content generated by using the trained image generation model obtained by training the target building image set better conforms to the characteristics of the building textures, and has better semantic consistency and geometric consistency, that is, the completed image content is more accurate, so that the difference between the completed image content and the image content actually lost in the missing area can be reduced, and the obtained target image has better visual effect.
In some embodiments, as shown in fig. 2, a flow chart of another image complement method is provided, and for ease of understanding, the following description is made with reference to fig. 2, in which:
step 201, a first image to be complemented is input.
And 202, extracting straight lines. And performing linear extraction processing on the first image to be complemented to obtain a global line penetrating through the missing area in the first image.
In some embodiments, the missing regions are segmented based on a global straight line, resulting in a plurality of sub-missing regions.
Step 203, mask blocking. The mask of the missing region is segmented based on the global straight line, and the mask of each sub-missing region is obtained.
Step 204, image complement. And combining the noise image of each sub-missing region with the first image to obtain a preliminary complement image.
The noise map of the sub-missing region is image content corresponding to the sub-missing region extracted from the target Gaussian noise map according to a mask of the sub-missing region. The target Gaussian noise map is one of the Gaussian noise maps which are obtained according with the number of the sub-missing regions when the first image is subjected to noise adding for a plurality of times.
Step 205, generating a target image. And performing noise reduction treatment on the preliminary complement image to obtain a target image.
In some embodiments, as shown in fig. 3, a flow chart of another image complement method is provided, which is applied to a computer device, and the trained image generation model is a trained diffusion model, and specifically includes the following steps:
1. the input first image 301 to be complemented is subjected to noise adding, and a first image 302 after noise adding is obtained. The first image 301 to be complemented includes a missing region where the image content is missing and a known region having the image content.
2. And performing an ANDed operation on the noisy first image 302 and the initial mask 303 of the first image to obtain a noisy image 304 of the known region. In some embodiments, the missing regions are segmented based on a global straight line, resulting in a plurality of sub-missing regions.
In some embodiments, where the missing region is at an image edge of the first image, the missing region is segmented based on global straight lines and edge features to obtain a plurality of sub-missing regions.
3. The first image 301 is denoised multiple times by the number of sub-missing regions, resulting in a gaussian noise map 305 that corresponds to the number. Wherein the gaussian noise in each gaussian noise plot is different.
4. Each gaussian noise map 305 is noise reduced, and each gaussian noise map 306 after noise reduction is obtained.
5. And performing an ANDed operation on each Gaussian noise map 306 after noise reduction and each mask 307 of the sub-missing region to obtain a noise map 308 of each sub-missing region after noise reduction.
As can be seen from fig. 3, the step of obtaining the mask 307 for each sub-absent area includes: performing inverse operation on the initial mask 303 of the first image to obtain a mask of the missing region; and partitioning the mask of the missing region based on the global straight line to obtain the mask of each sub-missing region.
6. The noise maps 308 of the respective sub-missing regions after noise reduction are combined to obtain a noise map 309 of the missing region.
7. And combining the noise map 309 of the missing region and the image 304 obtained by adding noise to the known region to obtain a preliminary complement image.
It will be appreciated that the preliminary complement image is an image obtained when the first image to be complemented is first complemented, and thus, an image obtained by combining the noise map 308 of each sub-missing region after noise reduction and the image 304 after noise addition to the known region is also the preliminary complement image.
8. Resampling the preliminary complement image results in a resampled image 310.
9. And carrying out iterative noise reduction processing on the resampled image 310 through the trained diffusion model to obtain a progressive complement image, and taking the progressive complement image after iteration stopping as a target image until iteration stopping. That is, the result of each noise reduction processing is used as the input of the next iteration, and then through multiple iterations, the target image with the image content in the missing region being completed and the image content in the missing region having semantic consistency and geometric consistency with the image content of the known region can be obtained.
It should be noted that, the mask m is usually a binary image, white represents an unknown region, and black represents a known region. When the truth image is x, the image content of the unknown region is expressed as m+x, and the image content of the known region is expressed as (1-m) +x. It will be appreciated that when the trained image generation model is a trained diffusion model, the inverse of the diffusion model proceeds from x in accordance with the theory of the Markov chain t To x t-1 Each step of (2) depends on x only t (intermediate image at time t in Markov chain of diffusion model), i.e. the known region (1-m) ≡x can be changed as long as the correctness of the corresponding distribution is ensured. The forward process of the diffusion model also applies noise to the samples in accordance with the theory of the Markov chain, so we can get intermediate results of the sampling at any time t.Is sampled from the input image, i.e. from a known region in the first image to be complemented, and +.>Is generated by a trained image generation model. Mask +.>And->Merging, utilizing resampling to facilitate generation of a further complement image x by a trained diffusion model t-1 Advanced complement image x after iteration stop t-1 Namely, the target image.
Wherein, the liquid crystal display device comprises a liquid crystal display device, Represents a posterior probability distribution function, and q represents an a priori probability distribution function. It will be appreciated that in the noise-adding of the first image 301 to be complemented, the noise-adding is performed using the prior probability distribution function q. When the resampled image 310 is iteratively noise reduced using a trained diffusion model, the representative posterior probability distribution function is used>To perform denoising.
In the image complement method, the missing areas are segmented by utilizing the linear information, and different Gaussian noises are added for different sub-missing areas to obtain the noise images of the different sub-missing areas, so that in the process of carrying out noise reduction processing on the preliminary complement image to obtain the target image, the complement image content matched with the sizes of the sub-missing areas is generated for each sub-missing area, the phenomenon of fitting is avoided, the generated complement image content is more accurate, the difference between the complement image content and the image content actually lost by the missing areas is reduced, and the obtained target image has better visual effect.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides an image complement device for realizing the image complement method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the image complement device provided below may be referred to the limitation of the image complement method hereinabove, and will not be repeated here.
In one embodiment, as shown in fig. 4, there is provided an image complement apparatus including: a straight line extraction module 401, a noise adding module 402, a combining module 403 and a noise reducing module 404, wherein:
the straight line extraction module 401 is configured to perform straight line extraction processing on a first image to be complemented to obtain straight line information; the first image comprises a missing area for missing image content; partitioning the missing region based on the linear information to obtain a plurality of sub-missing regions;
a denoising module 402, configured to denoise the first image multiple times according to the number of the sub-missing regions, so as to obtain gaussian noise diagrams according with the number; the gaussian noise in each gaussian noise plot is different;
a merging module 403, configured to select, for each sub-missing region, a target gaussian noise map from the gaussian noise maps that match the number of sub-missing regions, and extract image content corresponding to the sub-missing region from the target gaussian noise map, so as to obtain a noise map of the sub-missing region; combining the noise image of each sub-missing region with the first image to obtain a preliminary complement image;
And the noise reduction module 404 is configured to perform noise reduction processing on the preliminary complement image to obtain a target image.
In some embodiments, the first image includes a known region having image content therein; the straight line information includes a global straight line; the global straight line is a straight line that penetrates through the missing region in the first image; as shown in fig. 5, the straight line extraction module 401 includes a straight line extraction unit 401a, a cluster fusion unit 401b, and a selection unit 401c, wherein:
the line extraction unit 401a is configured to perform line extraction processing on a known region in the first image to be complemented, so as to obtain a local line in the known region.
The clustering fusion unit 401b is configured to perform clustering processing on each local straight line, and fuse each local straight line collected into one type, so as to obtain a plurality of candidate straight lines.
A selecting unit 401c, configured to select a global straight line from a plurality of candidate straight lines.
In some embodiments, the line extraction module 401 is configured to form a binary image based on the extracted global line, and obtain a line binary image; the linear region pixel value in the linear binary image is different from the non-linear region pixel value; and performing phase operation on the linear binary image and the missing region to block the missing region to obtain a plurality of sub-missing regions.
In some embodiments, the straight line extracting module 401 is specifically configured to extract an edge feature of the first image in a case where the missing region is at an image edge of the first image; performing phase-separating operation on the linear binary image and the missing region to divide the missing region so as to obtain a non-closed missing sub-region positioned at the edge of the image and a closed missing sub-region positioned at the edge of the non-image; and adding corresponding image edges to the non-closed deletion sub-regions based on the edge features to obtain closed deletion sub-regions positioned at the image edges.
In some embodiments, the combining module 403 is configured to determine an initial mask for the first image; performing inverse operation on the initial mask to obtain a mask of the missing region; partitioning the mask of the missing region based on the linear information to obtain masks of all sub-missing regions; and extracting image content corresponding to the sub-missing region from the target Gaussian noise map according to the mask of the sub-missing region to obtain a noise map of the sub-missing region.
In some embodiments, the noise reduction module 404 is configured to take the preliminary complement image as a current complement image, and perform noise reduction on the current complement image to obtain a further complement image; and taking the progressive complement image as a current complement image, returning to execute the step of carrying out noise reduction treatment on the current complement image to obtain the progressive complement image so as to carry out iterative noise reduction until iteration is stopped, and taking the progressive complement image after iteration is stopped as a target image.
The respective modules in the image complement apparatus described above may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an image complement method.
It will be appreciated by those skilled in the art that the structure shown in FIG. 6 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as Static Random access memory (Static Random access memory AccessMemory, SRAM) or dynamic Random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. A method of image complement, the method comprising:
performing linear extraction processing on a known region with image content in a first image to be complemented to obtain a local linear in the known region; the first image comprises a missing area for missing image content;
clustering the local straight lines, and fusing the local straight lines gathered into one type to obtain a plurality of candidate straight lines;
Selecting a global straight line from the plurality of candidate straight lines; the global straight line is a straight line that penetrates through a missing region in the first image; partitioning the missing region based on the global straight line to obtain a plurality of sub-missing regions;
carrying out noise adding on the first image for multiple times according to the number of the sub-missing areas to obtain Gaussian noise diagrams conforming to the number; the gaussian noise in each of the gaussian noise maps is different;
selecting a target Gaussian noise map from the Gaussian noise maps conforming to the number for each sub-missing region, extracting image content corresponding to the sub-missing region from the target Gaussian noise map, and obtaining a noise map of the sub-missing region;
noise reduction is carried out on the noise diagram of each sub-missing area;
noise is added to the first image;
combining the noise map of each noise-reduced sub-missing region with the first image after noise addition to obtain a preliminary complement image;
carrying out noise reduction treatment on the preliminary complement image through a trained image generation model to obtain a target image;
the trained image generation model is obtained by training a target building image set which is in different scenes and contains different building textures; the target building image in the target building image set is obtained by preprocessing an original building image; the step of preprocessing the original building image comprises the following steps:
Performing projection alignment on an original building image to obtain a projection alignment image of the original building image;
clipping the projection correction image to obtain a clipping image meeting the preset resolution condition;
and determining a target clipping image meeting the building texture semantics from the clipping images, and taking the target clipping image as a target building image.
2. The method of claim 1, wherein the partitioning the missing regions based on the global straight line results in a plurality of sub-missing regions, comprising:
forming a binary image based on the extracted global straight line to obtain a straight line binary image; the linear region pixel value in the linear binary image is different from the non-linear region pixel value;
and performing phase-phase operation on the linear binary image and the missing region to block the missing region to obtain a plurality of sub-missing regions.
3. The method of claim 2, wherein the performing an anding operation on the rectilinear binary image and the missing regions to block the missing regions to obtain a plurality of sub-missing regions comprises:
extracting edge features of the first image under the condition that the missing area is at the image edge of the first image;
Performing phase-separating operation on the linear binary image and the missing region to divide the missing region so as to obtain a non-closed missing sub-region positioned at the edge of the image and a closed missing sub-region positioned at the edge of the non-image;
and adding a corresponding image edge to the non-closed deletion sub-region based on the edge characteristic to obtain a closed deletion sub-region positioned at the image edge.
4. The method according to claim 1, wherein the method further comprises:
determining an initial mask of the first image;
performing inverse operation on the initial mask to obtain a mask of the missing region;
partitioning the mask of the missing region based on the global straight line to obtain masks of all the sub-missing regions;
the extracting the image content corresponding to the sub-missing region from the target Gaussian noise map to obtain the noise map of the sub-missing region comprises the following steps:
and extracting image content corresponding to the sub-missing region from the target Gaussian noise map according to the mask of the sub-missing region to obtain a noise map of the sub-missing region.
5. The method according to claim 1, wherein the denoising the preliminary complement image by the trained image generation model to obtain a target image comprises:
Taking the preliminary complement image as a current complement image, and carrying out noise reduction treatment on the current complement image through a trained image generation model to obtain a progressive complement image;
and taking the progressive complement image as the current complement image, returning to the step of executing the noise reduction processing on the current complement image through the trained image generation model to obtain the progressive complement image, carrying out iterative noise reduction until iteration is stopped, and taking the progressive complement image after iteration is stopped as a target image.
6. An image complement apparatus, the apparatus comprising:
the linear extraction module is used for carrying out linear extraction processing on a known region with image content in a first image to be complemented to obtain a local linear in the known region; the first image comprises a missing area for missing image content; clustering the local straight lines, and fusing the local straight lines gathered into one type to obtain a plurality of candidate straight lines; selecting a global straight line from the plurality of candidate straight lines; the global straight line is a straight line that penetrates through a missing region in the first image; partitioning the missing region based on the global straight line to obtain a plurality of sub-missing regions;
The noise adding module is used for adding noise to the first image for a plurality of times according to the number of the sub-missing areas to obtain Gaussian noise diagrams conforming to the number; the gaussian noise in each of the gaussian noise maps is different;
the merging module is used for selecting a target Gaussian noise map from the Gaussian noise maps conforming to the quantity for each sub-missing region, extracting image content corresponding to the sub-missing region from the target Gaussian noise map, and obtaining a noise map of the sub-missing region; noise reduction is carried out on the noise diagram of each sub-missing area; noise is added to the first image; combining the noise map of each noise-reduced sub-missing region with the first image after noise addition to obtain a preliminary complement image;
the noise reduction module is used for carrying out noise reduction processing on the preliminary complement image through the trained image generation model to obtain a target image; the trained image generation model is obtained by training a target building image set which is in different scenes and contains different building textures; the target building image in the target building image set is obtained by preprocessing an original building image; the step of preprocessing the original building image comprises the following steps: performing projection alignment on an original building image to obtain a projection alignment image of the original building image; clipping the projection correction image to obtain a clipping image meeting the preset resolution condition; and determining a target clipping image meeting the building texture semantics from the clipping images, and taking the target clipping image as a target building image.
7. The apparatus of claim 6, wherein the line extraction module is configured to form a binary image based on the extracted global line, and obtain a line binary image; the linear region pixel value in the linear binary image is different from the non-linear region pixel value; and performing phase-phase operation on the linear binary image and the missing region to block the missing region to obtain a plurality of sub-missing regions.
8. The apparatus according to claim 7, wherein the straight line extraction module is specifically configured to extract edge features of the first image if the missing region is at an image edge of the first image; performing phase-separating operation on the linear binary image and the missing region to divide the missing region so as to obtain a non-closed missing sub-region positioned at the edge of the image and a closed missing sub-region positioned at the edge of the non-image; and adding a corresponding image edge to the non-closed deletion sub-region based on the edge characteristic to obtain a closed deletion sub-region positioned at the image edge.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310513801.7A CN116228605B (en) | 2023-05-09 | 2023-05-09 | Image complement method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310513801.7A CN116228605B (en) | 2023-05-09 | 2023-05-09 | Image complement method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116228605A CN116228605A (en) | 2023-06-06 |
| CN116228605B true CN116228605B (en) | 2023-09-08 |
Family
ID=86580978
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310513801.7A Active CN116228605B (en) | 2023-05-09 | 2023-05-09 | Image complement method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116228605B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101192269A (en) * | 2006-11-29 | 2008-06-04 | 佳能株式会社 | Method and device for estimating vanishing point from image, computer program and its storage medium |
| JP2013110529A (en) * | 2011-11-18 | 2013-06-06 | Samsung Yokohama Research Institute Co Ltd | Image processing device, image processing method, and program |
| CN105825478A (en) * | 2015-01-26 | 2016-08-03 | 索尼公司 | Structure analysis method for recovering missing structures in an image after object removal |
| CN108765315A (en) * | 2018-05-04 | 2018-11-06 | Oppo广东移动通信有限公司 | Image completion method, apparatus, computer equipment and storage medium |
| CN111583159A (en) * | 2020-05-29 | 2020-08-25 | 北京金山云网络技术有限公司 | Image completion method and device and electronic equipment |
| CN115829865A (en) * | 2022-11-16 | 2023-03-21 | 百果园技术(新加坡)有限公司 | Image completion method, system, device and storage medium based on model prior |
| US11636578B1 (en) * | 2020-05-15 | 2023-04-25 | Apple Inc. | Partial image completion |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4266241A1 (en) * | 2021-05-18 | 2023-10-25 | Samsung Electronics Co., Ltd. | Electronic device for performing image inpainting, and method for operating same |
-
2023
- 2023-05-09 CN CN202310513801.7A patent/CN116228605B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101192269A (en) * | 2006-11-29 | 2008-06-04 | 佳能株式会社 | Method and device for estimating vanishing point from image, computer program and its storage medium |
| JP2013110529A (en) * | 2011-11-18 | 2013-06-06 | Samsung Yokohama Research Institute Co Ltd | Image processing device, image processing method, and program |
| CN105825478A (en) * | 2015-01-26 | 2016-08-03 | 索尼公司 | Structure analysis method for recovering missing structures in an image after object removal |
| CN108765315A (en) * | 2018-05-04 | 2018-11-06 | Oppo广东移动通信有限公司 | Image completion method, apparatus, computer equipment and storage medium |
| US11636578B1 (en) * | 2020-05-15 | 2023-04-25 | Apple Inc. | Partial image completion |
| CN111583159A (en) * | 2020-05-29 | 2020-08-25 | 北京金山云网络技术有限公司 | Image completion method and device and electronic equipment |
| CN115829865A (en) * | 2022-11-16 | 2023-03-21 | 百果园技术(新加坡)有限公司 | Image completion method, system, device and storage medium based on model prior |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116228605A (en) | 2023-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Singh et al. | Single image dehazing using gradient channel prior | |
| Zhang et al. | Multi-scale single image dehazing using perceptual pyramid deep network | |
| Yang | Hardware-efficient bilateral filtering for stereo matching | |
| Lu et al. | Deep texture and structure aware filtering network for image smoothing | |
| DE102016011380A1 (en) | Image synthesis using an active mask | |
| Cao et al. | New architecture of deep recursive convolution networks for super-resolution | |
| CN113793284B (en) | Image restoration method for nonuniform blocking of aerodynamic optical effect space-variant blurred image | |
| Ting et al. | Image inpainting by global structure and texture propagation | |
| Pandey et al. | Classification of priors and regularization techniques appurtenant to single image super-resolution | |
| CN113935925B (en) | Pneumatic optical effect space-variant blurred image restoration method and system | |
| CN116612280A (en) | Vehicle segmentation method, device, computer equipment and computer readable storage medium | |
| Franke et al. | Vet: Visual error tomography for point cloud completion and high-quality neural rendering | |
| CN116563303B (en) | Scene generalizable interactive radiation field segmentation method | |
| Bu et al. | Linear recursive non-local edge-aware filter | |
| Fang et al. | Learning explicit smoothing kernels for joint image filtering | |
| CN116228605B (en) | Image complement method, device, computer equipment and storage medium | |
| Kottler et al. | 3GAN: A Three-GAN-based Approach for Image Inpainting Applied to the Reconstruction of Occluded Parts of Building Walls. | |
| Fattal et al. | Edge-based image coarsening | |
| Qiang et al. | Exemplar-based pixel by pixel inpainting based on patch shift | |
| CN115630660B (en) | Barcode positioning method and device based on convolutional neural network | |
| Jung et al. | Intensity-guided edge-preserving depth upsampling through weighted L0 gradient minimization | |
| Li et al. | Efficient spatially sparse inference for conditional gans and diffusion models | |
| Ahmed et al. | Digital image inpainting techniques for cultural heritage preservation and restoration | |
| Vlašánek et al. | Patch based inpainting inspired by the F1-transform | |
| Jang et al. | PUCA: patch-unshuffle and channel attention for enhanced self-supervised image denoising |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |