CN107507216A - The replacement method of regional area, device and storage medium in image - Google Patents
The replacement method of regional area, device and storage medium in image Download PDFInfo
- Publication number
- CN107507216A CN107507216A CN201710706278.4A CN201710706278A CN107507216A CN 107507216 A CN107507216 A CN 107507216A CN 201710706278 A CN201710706278 A CN 201710706278A CN 107507216 A CN107507216 A CN 107507216A
- Authority
- CN
- China
- Prior art keywords
- area
- image
- characteristic point
- deformation
- replacement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 132
- 230000008569 process Effects 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 30
- 230000011218 segmentation Effects 0.000 claims abstract description 24
- 238000001514 detection method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 230000007613 environmental effect Effects 0.000 claims description 8
- 238000007689 inspection Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 230000008859 change Effects 0.000 abstract description 22
- 230000015572 biosynthetic process Effects 0.000 abstract description 4
- 238000003786 synthesis reaction Methods 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 15
- 230000009466 transformation Effects 0.000 description 15
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 12
- 238000000605 extraction Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000001815 facial effect Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 101100295776 Drosophila melanogaster onecut gene Proteins 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 210000004709 eyebrow Anatomy 0.000 description 2
- 210000000887 face Anatomy 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 239000000729 antidote Substances 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
The invention provides replacement method, device and the storage medium of regional area in a kind of image, replacement method includes the first area obtained respectively in benchmark image and replaces the characteristic point of second area and the positional information of characteristic point in image；Foreground segmentation process is carried out to first area and second area；According to the positional information of characteristic point, first, global deformation process is carried out to replacing image so that second area in replacement image and the size of first area and the difference of position after deformation are in preset range；Local deformation processing then is carried out to the first area in benchmark image and its neighboring region；First area after deformation is replaced with into the second area after deformation, obtains the benchmark image for including second area.On the premise of the replacement process of the present invention can not change ensureing the feature for being replaced image, minimally change the benchmark image for replacement, improve the naturalization degree of image after replacing, and cause naked eyes to look no the vestige of synthesis.
Description
Technical field
The present invention relates to technical field of image processing, and in particular to the replacement method of regional area, device in a kind of image
And storage medium.
Background technology
With the development of the intelligent equipment in today's society, image procossing have become people life and in it is indispensable
A part, the professional image in either working still live in amusement type image procossing, most important of which figure
As one of processing mode be exactly the regional area in image is replaced, with obtain work or amusement needed for replacement after figure
Picture so that replace image processing mode can be used in the accelerated selection to virtual experience and save image synthesis cost with
And the entertaining synthesis of enhancing image is recreational.
At present, mainly there are two kinds to the mode that the regional area in image is replaced, first way is to pass through extraction
The set of characteristic points of first area image and correspondingly " stingy graph region image "；Calculate the set of characteristic points in second area image
With corresponding " textures area image "；According to the parameter adjustment second area image of the set of characteristic points of the first area image
" textures area image ", obtain " replace textures area image "；" textures area image will be replaced " and replace " the Kou Tu areas
Area image "；The second way is the key position of identification region, including the key of the key position of first area and second area
Position；The key position of key position and second area to first area positions；Calculate the key position of second area
To the motion vector field of the key position of first area；Second area is deformed to by first area according to the motion vector field
Position deformed after region；Naturalization processing is carried out to the region after the deformation.
However, all there occurs the change on geometry on the area image finally used for both above-mentioned modes；Make
The picture shape somewhat difference for obtaining first area and second area will be unable to recognize used in the image finally synthesized
Content is provided by which region, and it is a basic demand to keep information not change in some applications；Due to depositing
In such technological deficiency, such method is caused to be simply possible to use in recreational application.
The content of the invention
For the problems of the prior art, the present invention provides the replacement method of regional area, device in a kind of image and deposited
Storage media, rapidly and reliably property is high for replacement process, on the premise of can not changed ensureing the feature for being replaced image, most
Change to small degree the benchmark image for replacement, improve the accuracy of the replacement of regional area.
In order to solve the above technical problems, the present invention provides following technical scheme：
In a first aspect, the present invention provides a kind of replacement method of regional area in image, the replacement method includes：
The first area in benchmark image is obtained respectively and replaces the characteristic point and characteristic point of the second area in image
Positional information；Wherein, area attribute of the first area in the benchmark image and the second area are in the benchmark
Area attribute in image is identical；
The first area and second area are carried out at foreground segmentation respectively in the benchmark image and replacement image
Reason；
According to the first area and the positional information of the characteristic point of second area, global shape is carried out to the replacement image
Change is handled so that the size and position of the size and position of replacing the second area in image with the first area after deformation
Difference be in preset range；
According to the first area after deformation and the positional information of the characteristic point of second area, in the benchmark image
One region and its neighboring region carry out local deformation processing so that the first area and the geometry phase of the second area
Together；Wherein, the neighboring region is in the extended area centered on the first area and adjacent with the first area
Region；
And the first area after deformation is replaced with into the second area after deformation, obtain including the second area
Benchmark image.
Further, the first area obtained respectively in benchmark image and the feature for replacing the second area in image
The positional information of point and characteristic point, including：
Target area detection of attribute is carried out to the benchmark image and replacement image respectively, and extracts firstth area respectively
The positional information of each characteristic point and each characteristic point of domain and second area.
Further, the spy of the second area in the first area and replacement image in the benchmark image of acquisition respectively
After the positional information of sign point and characteristic point, methods described also includes：
According to the first area and the characteristic point of second area and the positional information of characteristic point in image are replaced, is judged
The first area be used for examine attribute information with corresponding second area be used for inspection attribute information whether
In default range of attributes, that is, judge whether the replacement image passes through inspection；
If so, foreground segmentation then is carried out to the first area and second area in the benchmark image and replacement image
Processing；
Otherwise, it is determined that the second area currently replaced in image can not replace the first area in benchmark image, and export
What can not be replaced informs prompting.
Further, it is described before the benchmark image and replacing is carried out to the first area and second area in image
Scape dividing processing, including：
According to the structural relation between each characteristic point of the first area, in the benchmark image by first area with
Environmental background is split, and obtains the mask of the first area and the first area；
And according to the structural relation between each characteristic point of the second area, by second in the replacement image
Region is split with environmental background, obtains the mask of the second area and the second area.
Further, it is described according to the first area and the positional information of the characteristic point of second area, to the replacement
Image carries out global deformation process so that the size for replacing the second area in image and position and firstth area after deformation
The size in domain and the difference of position are in preset range, including：
According to the first area and the characteristic point of second area, geometric deformation mould is established using global distortion removal method
Type；
Global deformation process is carried out to the replacement image based on the geometric deformation model
And the deformation results according to the replacement image, the position letter of the corresponding characteristic point for adjusting the second area
The mask of breath and the second area.
Further, in the first area according to after deformation and the positional information of the characteristic point of second area, to institute
State the first area in benchmark image and its neighboring region carries out local deformation processing so that the first area and described second
Before the geometry in region is identical, in addition to：
The position relationship network of the characteristic point of first area is built according to the positional information of N number of characteristic point of first area；
And the position relationship network of the characteristic point of second area is built according to the positional information of N number of characteristic point of second area；
And the position relationship network of the first area is finely divided, obtain M characteristic point so that described first
The characteristic point sum in region is M+N；And the position relationship network of the second area is finely divided, obtain M feature
Point so that the characteristic point sum of the second area is also M+N；
Wherein, M and N is positive integer.
Further, the positional information of the characteristic point of the first area and second area according to after deformation, to described
First area and its neighboring region in benchmark image carry out local deformation processing so that the first area and secondth area
The geometry in domain is identical, including：
Outside first area and each characteristic point for replacing the second area in image in the benchmark image respectively
Protecting frame is set, wherein, the position of the protecting frame is built according to the central point of protecting frame, wherein, the central point of protecting frame is
The center in the closing convex closure region being made up of the circumference of whole characteristic points；
And according to the first area after deformation and the positional information of the characteristic point of second area, in the benchmark image
In first area each characteristic point outside protecting frame in, at the carry out deformation of the first human face region controlled each characteristic point
Reason so that the first area is identical with the geometry of the second area.
Further, the first area by after deformation replaces with the second area after deformation, obtains including described
The benchmark image in two regions, including：
First area after deformation is replaced with into the second area after deformation, and the region replaced is the firstth area after deformation
The intersection area of the mask of second area after the mask in domain and deformation, obtain the benchmark image for including the second area；
And naturalization fusion treatment is carried out to the second area in the benchmark image including the second area.
Second aspect, the present invention also provide the alternative of regional area in image a kind of, including memory, processor and
The computer program that can be run on a memory and on a processor is stored, it is real during computer program described in the computing device
In existing described image the step of the replacement method of regional area.
The third aspect, the present invention also provide a kind of computer-readable recording medium, are stored thereon with computer program, the meter
The step of calculation machine program realizes the replacement method of regional area in described image when being executed by processor.
As shown from the above technical solution, the replacement method of regional area, device and deposited in a kind of image provided by the invention
Storage media, replacement method include the first area obtained respectively in benchmark image and the characteristic point for replacing the second area in image
And the positional information of characteristic point；Foreground segmentation process is carried out to first area and second area；According to first area and the secondth area
The characteristic point in domain, first, global deformation process is carried out to replacing image so that the second area replaced in image after deformation
The difference of the size and position of size and position and first area is in preset range；According to the first area after deformation and
The positional information of the characteristic point in two regions, deformation process then is carried out to the first area in benchmark image so that first area
It is identical with the geometry of second area；And the first area after deformation is replaced with into the second area after deformation, wrapped
Include the benchmark image of second area.Rapidly and reliably property is high for the replacement process of the present invention, and the spy of image can be replaced in guarantee
On the premise of sign does not change, minimally change the benchmark image for replacement, improve the replacement of regional area
The naturalization degree of image after accuracy and replacement, and cause naked eyes to look no the vestige of synthesis, after replacement being ensured
Regional area do not change, and solve in image in regional area replacement technology outside regional area deformation and regional area
Composograph invalidity problem caused by enclosing gross distortion.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are the present invention
Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
These accompanying drawings obtain other accompanying drawings.
Fig. 1 is the schematic flow sheet of the replacement method of regional area in a kind of image of the invention；
Fig. 2 is the schematic flow sheet of step 100 in the replacement method of regional area in image of the invention；
Fig. 3 be the present invention image in regional area replacement method in step A01 and step A02 schematic flow sheet；
Fig. 4 is the journey schematic diagram of step 300 in the replacement method of regional area in image of the invention；
Fig. 5 is the schematic flow sheet of step 400 in the replacement method of regional area in image of the invention；
Fig. 6 is the schematic flow sheet of step 500 in the replacement method of regional area in image of the invention；
Fig. 7 is the schematic flow sheet of the replacement method of face in the image in application example of the present invention；
Fig. 8 is that face replaces the method for calibration of feasibility in the replacement method of face in the image in application example of the present invention
Schematic flow sheet；
Fig. 9 is the structural representation of the replacement system of regional area in a kind of image of the invention.
Embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, clear, complete description is carried out to the technical scheme in the embodiment of the present invention, it is clear that described embodiment is
Part of the embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.
The defects of image that user's face of the prior art deforms upon is replaced present in case is directed to, the present invention carries
A kind of replacement method of regional area in image is supplied, the advantage of replacement method of the invention, which essentially consists in, keeps user's shape of face not
Change, thus effect looks more natural and can picked out or user, and the replacement method of the application and
The mode that 3D is transformed into from 2D is compared and realizes saving in computing resource, while the feature that can ensure to be replaced image is not sent out
Changing.
Embodiments of the invention one provide a kind of a kind of embodiment of the replacement method of regional area in image,
Referring to Fig. 1, the replacement method of regional area specifically includes following content in described image：
Step 100：Respectively obtain benchmark image in first area and replace image in second area characteristic point and
The positional information of characteristic point；Wherein, area attribute of the first area in the benchmark image exists with the second area
Area attribute in the benchmark image is identical.
In step 100, the processor for being provided with characteristic point acquiring unit obtains first area in benchmark image respectively
With the characteristic point of second area and the positional information of characteristic point in replacement image；It will be appreciated that it is provided with characteristic point acquisition
The background area of image and secondth area replaced in image on the basis of the image that the processor of unit obtains after finally replacing
Domain.Wherein, area attribute of the first area in the benchmark image and the second area are in the benchmark image
Area attribute it is identical；The area attribute can show that the area type of first area and second area in the picture, in reality
In the application of border, the first area to be specifically replaced and the area attribute of second area, it can select to set by user or answer
Determined with pre-setting.
For example, if current base image and replacing in image and including portrait, first area and second area
Area attribute can be the region parts such as the hand of same modality, face, face, leg, extremity portion, neck, that is to say, that
Region of area attribute of the first area in the benchmark image with the second area in the replacement image belongs to
Property it is identical be：If first area refers to the hand of the face forward the five fingers expansion of portrait in benchmark image, then second area
As replace the hand that the face forward the five fingers deploy in image.
In another citing, if including plant, first area and second in current base image and replacement image
The area attribute in region can be the region parts such as the stem of same modality, leaf, flower, limb, that is to say, that the first area
Area attribute of the area attribute with the second area in the replacement image in the benchmark image is identical as：Such as
If first area is referred in benchmark image, wide-open flower, second area are to replace plant in image in front of plant side
Wide-open flower in front of side.
In addition to attribute is identical, also need to extract characteristic point and the characteristic point position in the first and second regions in step 100
Confidence ceases, positional information here can be comprising relevant location informations such as center of gravity, critical component, profiles, for hand region,
Positional information can be set as the outline information of five finger fingertip positions and hand, the plant for same modality
Strain, positional information can be set as regional center and the outline position of key position (such as each petal).
In order to which the first area in benchmark image is accurately replaced with into second area, the of benchmark image is being determined
One region and replace after the characteristic point of second area and the positional information of characteristic point in image, it is necessary to by first area from benchmark
Distinguished in the environmental background of image, and first area is also distinguished from the environmental background for replacing image, so
First area and second area can accurately handle and replace.Step 200：In the benchmark image and replacement image
Foreground segmentation process is carried out to the first area and second area.
In step 200, the processor of dividing processing unit is provided with the benchmark image and is replaced in image to institute
State first area and second area carries out foreground segmentation process.It is understood that can be according to the structure between each characteristic point
First area described in relation pair and second area carry out foreground segmentation process.For example, if the first area and the secondth area
Domain is face, then the characteristic point of face then includes eyebrow, eyes, nose, mouth, the position etc. of face mask, therefore, face
Structural relation between characteristic point can obtain according to the priori of human face structure, and the priori of human face structure is face
The time-space relationship of muscle interaction, i.e., determined by the anatomical structure of face, do not influenceed by imaging circumstances.
Because the profile of first area and the profile of second area are normally present difference, and the firstth area of benchmark image
Domain generally corresponds to need commodity or the service marketed, therefore the foreground segmentation process in through step 200 obtains reference map
, it is necessary to be carried out to the second area replaced in image complete on profile after second area in the first area and replacement image of picture
Office's adjustment, to eliminate the difference between the profile for replacing the second area and first area in image.
Step 300：According to the first area and the positional information of the characteristic point of second area, the replacement image is entered
The global deformation process of row so that the chi of the size for replacing the second area in image and position and the first area after deformation
The very little and difference of position is in preset range.
In step 300, the processor for replacing image deformation unit is provided with according to the first area and second area
Characteristic point positional information, global deformation process is carried out to the replacement image so that replace in image the after deformation
The size in two regions and position are in preset range with the size of the first area and the difference of position.It is appreciated that
It is that the preset range determines according to practical situation in actual applications, and the principle that the preset range is set makes to try one's best
The size for obtaining second area is consistent with the size of the first area.By taking face as an example, in order to ensure that size is consistent with position,
According to benchmark image and the key coordinate of face in image can be replaced by rigid body translation (Rigid Transformation),
Using least square method and the constant algorithm of stochastical sampling (RANSAC), transformation matrix is calculated, and then image mapping will be replaced
Into benchmark image coordinate system.Step 400：According to the first area after deformation and the positional information of the characteristic point of second area,
Local deformation processing is carried out to the first area in the benchmark image and its neighboring region so that the first area with it is described
The geometry of second area is identical；Wherein, the neighboring region for it is in extended area centered on the first area,
And the region adjacent with the first area.
Wherein, the extended area can be arranged to arbitrary shape according to practical situations, for example, the extended area
For rectangular area or border circular areas etc..
In step 400, after the second area to replacing in image carries out the global adaptation on profile, to improve first
The accuracy replaced between region and second area, it is also necessary to local directed complete set is carried out to the first area in benchmark image, that is, set
The processor of benchmark image deformation unit is equipped with according to the first area after deformation and the positional information of the characteristic point of second area,
Local deformation processing is carried out to the first area in the benchmark image and its neighboring region so that the first area with it is described
The geometry of second area is identical.It is to be appreciated that this step is the important step of the application, it can simultaneously realize and replace
Journey is quick and the accurate effect of image after replacing.Wherein, geometry refers to the position where appearance profile and characteristic point；It is and several
What structure it is identical including：The first area is identical with the appearance profile of the second area, a certain in the first area
Position where characteristic point characteristic point corresponding with the second area is respectively positioned on a default model for being directed to this feature point
In enclosing；Specifically, if the first area and the second area are face, the face of the first area it is a certain
Position where characteristic point eyes characteristic point eyes corresponding with the face of the second area is directed to this feature positioned at one
In the default scope of point eyes；This can be set to 3mm for the default scope of this feature point eyes, i.e., described first area
Face a certain characteristic point eyes characteristic point eyes corresponding with the face of the second area where position coordinates
Difference is less than or equal to 3mm.
Step 500：First area after deformation is replaced with into the second area after deformation, obtains including the second area
Benchmark image.
In step 500, behind the first area after the second area and local deformation after obtaining global deformation, it is provided with
The processor of regional area replacement unit replaces with the first area after deformation the second area after deformation, obtains including described
The benchmark image of second area.Here, suppose that benchmark image is P, replacement image is Q, and region to be replaced is S, non-replaced region
For NS, the image finally synthesized is P, needs to meet
P ' (S)=Q (S)
P ' (NS)=P (NS)
It was found from foregoing description, the replacement process of the replacement method of regional area is fast in the image in embodiments of the invention
Speed and reliability height, and the accuracy of the replacement of regional area is improved, it can ensure that the regional area after replacing does not change
Become, solve in image and synthesized in regional area replacement technology caused by regional area deformation and local area periphery gross distortion
Image invalidity problem, while reduce the time-consuming and cost for replacing processing.
One kind that embodiments of the invention two provide step 100 in the replacement method of regional area in above-mentioned image is specific
Embodiment, referring to Fig. 2, the step 100 specifically includes following content：
Step 101：Target area detection of attribute is carried out to the benchmark image and replacement image respectively, determines described first
Region and the positional information of second area.
Step 102：Each characteristic point of the first area and second area and the positional information of each characteristic point are extracted respectively.
In a step 102, the operations such as the feature extraction of usual model, foreground segmentation are just completed when making database
, the response speed so handled faster, of course simultaneously processing can also, simply need more computing resource, algorithm is placed on this
Extraction process is not needed then during ground processing.It is determined that after the positional information of the first area and second area, the characteristic point
Positional information be position coordinates of this feature point in region.It is understood that first to input reference image and
Replace the feature point detection that image carries out the first area and second area.The basis of feature point detection is advanced pedestrian's face
Detection is with positioning datum image and replaces image position.
For example, if first area and second area are positive and unobstructed face, presently disclosed a large amount of faces
Detection algorithm (AdaBoost, NPD, MTCNN etc.) can be achieved accurately to detect face.After completing Face datection, to detection
The face arrived carries out feature point extraction, and the method for feature point extraction includes but is not limited to be based on deep learning (such as depth own coding
Network, depth Recurrent networks etc.) and method based on active contour model (such as ASM, AAM etc.).
It was found from foregoing description, in the image of embodiments of the invention in the replacement method of regional area, there is provided a kind of
Characteristic point and characteristic point reliable and that quickly obtain the first area in benchmark image and the second area in replacement image
The method of positional information, and the degree of accuracy of the characteristic point and the positional information of characteristic point obtained is high, for replacing for subsequent local area
Change and provide accurate data basis.
The step of embodiments of the invention three are provided in above-mentioned image in the replacement method of regional area after step 100
A kind of A01 and step A02 embodiment, referring to Fig. 3, the step A00 specifically includes following content：
Step A01：According to the first area and replace the characteristic point of second area in image and the position of characteristic point
Information, judge that the attribute information for being used to examine of the first area is believed with the attribute for being used to examine in corresponding second area
Breath judges whether the replacement image passes through inspection whether in default range of attributes.
It is understood that according to the first area and replace the characteristic point and characteristic point of the second area in image
Positional information, the attribute information to be tested of the first area and second area is judged whether in tolerance interval, that is,
Say and judge to replace whether image passes through verification.
If so, then enter step 200, i.e., to the first area and the secondth area in the benchmark image and replacement image
Domain carries out foreground segmentation process.
Otherwise, then into step A02.
Step A02：Judge that the current second area replaced in image can not replace the first area in benchmark image, and it is defeated
Go out can not replace inform prompting.It is understood that the second area in current replacement image is judged can not replace benchmark
Behind first area in image, what output can not replace informs prompting so that user knows respective feedback information.
It was found from foregoing description, in the image of embodiments of the invention in the replacement method of regional area, there is provided a kind of
The method of calibration of the replaceability of reliable and quick regional area, if verifying successfully, then carry out replacing for regional area in image
Change, and then ensure that the accuracy and availability of the replacement result of subsequent local area, improve the practicality of whole replacement method
Property and reliability.
In a kind of embodiment, one kind of step 200 is specific in the replacement method of regional area in above-mentioned image
Embodiment.The setting target of the step 200 is the structural relation between each characteristic point according to the first area, in institute
State in benchmark image and split first area and environmental background, obtain the mask of the first area and the first area,
And according to the structural relation between each characteristic point of the second area, by second area and ring in the replacement image
Border background is split, and obtains the mask of the second area and the second area, and it specifically includes following content：
On the whole, the step 200 includes：
(1) generation foreground and background area to contract and is expanded outwardly according to the profile point of first area or second area
Domain；Obtain the first doubtful foreground and background mask.
(2) mask being previously obtained and original image are split using oneCut algorithms, further refine background and
Foreground area, obtain the second doubtful foreground and background mask.
(3) finally, foreground area is handled using morphology (including opening operation and profile repairing, fill depression area
Domain), obtain final foreground and background mask.
Specifically, step 201：Doubtful foreground and background region initialization.Here auxiliary information is the face of face
Key point positional information, although having obtained the outline position of face in step 100, these positional information degrees of accuracy are not
It is enough, can not directly generate segmentation result, but can by the way that face mask to contract and is expanded outwardly, obtain doubtful prospect and
Background area is as initial segmentation region.Specifically, according between the first area and each characteristic point of second area
Structural relation, first area and environmental background are subjected to initial segmentation in the benchmark image, and in the replacement image
Second area and environmental background are subjected to initial segmentation.
Step 202：On the basis of primary segmentation can utilize many efficiently and accuratelies dividing method (such as graphCut,
GrabCut, oneCut, ACM etc.) carry out secondary Accurate Segmentation.A kind of preferable scheme is one cut in grabcut algorithms,
Algorithm is cut different from traditional figure, for the algorithm without iteration, segmentation can be achieved in once-through operation, and then obtains foreground area；
Step 203：Post-process the foreground area optimized, including gray value limitation and Morphological scale-space (opening and closing operation
With filling hole), on the one hand ensure face area grey similarity, on the other hand ensure the integrality of face mask.
In a kind of embodiment, the step 300 of the application specifically may be summarized to be following content to step 400：
(1) integral position, direction and size change over are carried out to replacing image；
(2) benchmark image local geometric is converted；
(3) in order to obtain naturally replacing result, first area and its neighboring region are also adjusted accordingly.
Specifically implementation process is described as follows described in example IV and the content of embodiment five：
One kind that embodiments of the invention four provide step 300 in the replacement method of regional area in above-mentioned image is specific
Embodiment, referring to Fig. 4, the step 300 specifically includes following content：
Step 301：According to the first area and the characteristic point of second area, using imitative in global distortion removal method
The mode for penetrating conversion establishes geometric deformation model.
It is understood that the mode of affine transformation is not the unique translation mode used in the present embodiment, global deformation
Any-mode in antidote can realize the present embodiment step 301 to the conversion process of step 303.
Step 302：Based on the geometric deformation model, the characteristic point for determining the second area according to least square method becomes
Change to the affine transformation parameter of the characteristic point of first area.
Step 303：Deformation is carried out to the replacement image according to the affine transformation parameter so that the replacement figure after deformation
The size of second area as in and position are in preset range with the size of the first area and the difference of position.
Step 304：According to the deformation results of the replacement image, the position of the corresponding characteristic point for adjusting the second area
The mask of information and the second area.
In above-mentioned steps 301 to 304, image alignment benchmark image will be replaced using the characteristic point extracted.Such as profit
With first area characteristic point and second area characteristic point, and by the use of affine transformation as geometric deformation model, pass through least square method
Estimation is from second area characteristic point to the affine transformation parameter of first area feature point transformation.And utilize the affine transformation parameter pair
Replace image and carry out the replacement that deformation acquisition is identical with benchmark image size, and the position of first area and second area matches substantially
Image.It is also required to replacing while image carries out deformation with identical parameter to mask and the feature point coordinates of second area
Carry out deformation.
It was found from foregoing description, in the image of embodiments of the invention in the replacement method of regional area, there is provided a kind of
Reliably and quickly according to the first area and the characteristic point of second area, deformation process side is carried out to the replacement image
Method, and then improve the practicality and reliability of whole replacement method.
One kind that embodiments of the invention five provide step 400 in the replacement method of regional area in above-mentioned image is specific
Embodiment, referring to Fig. 5, the step 400 concrete application triangulation network construction method：(1) Feature Points Matching so that first area
Corresponded with the key point of second area；(2) triangulation network is built, and using Delaunay scheduling algorithms, establishes the netted pass of characteristic point
System's figure；(3) loop subdivision algorithms are utilized, to being previously obtained network of personal connections refinement, build the dense triangulation network；(4) it is right
Each subgraph in the network relation figure of structure carries out affine parameter estimation and local interpolation respectively so that first area after conversion
Characteristic point matched completely with the characteristic point position information of second area, specifically include following content：
Step 401：The characteristic point of first area and second area is matched so that the characteristic point institute table of identical label
The attribute reached is identical.
Step 402：In addition to key point inside first area and second area and profile point, it is also necessary to which increase is outside to be controlled
Point, ensure first area to second area change when, target area and its neighborhood linkage change.
Specifically, the specific set-up mode of extended area is：All characteristic points can just be included by being calculated first
The positive rectangle of restriction；Then according to limiting the ratio of positive rectangle, positive rectangle will be limited to external expansion, the rectangular extension area is obtained
Domain.
Wherein, multiple quantiles on four summits and four edges of the rectangle after extension are to be used to produce extended area
And the characteristic point newly increased.
That is, just ensure human face region and the geometry of peripheral background by adds additional some protective features points
Structure does not change, and to the region in the range of protection zone only there occurs trickle geometric deformation beyond human face region,
And this geometric deformation is weakened by the triangle gridding of refinement so that visually looks and varies less and almost it is difficult to discover.
Step 403：The triangulation network is built using Delaunay algorithms, spatial relation is built between different characteristic point.
The position relationship network of the characteristic point of first area is built according to the positional information of N number of characteristic point of first area；And according to
The position relationship network of the characteristic point of the positional information structure second area of N number of characteristic point of second area.
Step 404：To ensure the follow-up naturality for replacing effect, using Loop subdivison algorithms, in original triangle
On the basis of net, the denser triangulation network of structure.Specifically, the position relationship network of the first area can be carried out thin
Point, obtain M characteristic point so that the characteristic point sum of the first area is M+N；And the position to the second area
Put relational network to be finely divided, obtain M characteristic point so that the characteristic point sum of the second area is M+N；Wherein, N and
M is positive integer.
Step 405：After triangulation network structure is completed, benchmark image can be directed to and replaced in image three corresponding to each subnet
The position structure affine Transform Model of angle point, make it that the position of benchmark image and replacement image subnet corresponding region is complete by projection
It is complete consistent.
In step 401 to 405, using the relative coordinate of characteristic point, more characteristic points are generated, by second area geometry
The face deformation of model's image is carried out the face geometry one with second area by the feature point coordinates after conversion as benchmark
Cause.The protecting frame can be circular frame or rectangle frame, and the position of the protecting frame is built according to the central point of protecting frame, its
In, the central point of protecting frame is the center in the closing convex closure region being made up of the circumference of whole characteristic points.
It was found from foregoing description, in the image of embodiments of the invention in the replacement method of regional area, there is provided a kind of
Reliably and quickly according to the first area after deformation and the positional information of the characteristic point of second area, in the benchmark image
The first area method that carries out deformation process, the accuracy of the replacement of regional area is improved, after it can ensure to replace
Regional area does not change, and regional area deformation and local area periphery are serious in regional area replacement technology in solution image
Composograph invalidity problem caused by deformation.
One kind that embodiments of the invention six provide step 500 in the replacement method of regional area in above-mentioned image is specific
Embodiment, referring to Fig. 6, the step 500 specifically includes following content：
Step 501：First area after deformation is replaced with into the second area after deformation, and the region replaced is after deformation
First area mask and deformation after second area mask intersection area, obtain the benchmark for including the second area
Image.
Step 502：The second area in benchmark image including the second area is carried out at naturalization fusion
Reason.
The first area of benchmark image after deformation is replaced using the second area, includes the second area after replacement
Benchmark image in second area be first area after two width deformation and second area mask common factor；And to including described
Second area in the benchmark image of second area carries out naturalization processing so that the benchmark image including the second area
Color is consistent.A kind of feasible method is Poisson editing Poisson editor's algorithms, and the algorithm is a kind of nothing of excellent performance
Stitching algorithm is stitched, the method by constructing Poisson equation solution pixel optimal values, is remaining the same of source images gradient information
When, it can be very good to merge the background of source images and target image.Simultaneously in order to accelerate to merge speed, input picture is comprising the
Minimum enclosed rectangle including one region and second area.
It was found from foregoing description, in the image of embodiments of the invention in the replacement method of regional area, there is provided a kind of
Reliably and the first area after deformation is quickly replaced with into the second area after deformation, obtain the base for including the second area
The method of quasi- image, and then improve the practicality and reliability of whole replacement method.
It is understood that processor described above can be a kind of electronic equipment provided with software program, can also
For a kind of pre- application program mounted in device end, can also be a kind of pre- function mould in the application program of device end
Block, wherein, device end includes but is not limited to mobile phone, tablet personal computer, PC or embedded system etc..
It is understood that the realization order of the step in above-mentioned image in the replacement method of regional area is answered according to actual
Any adjustment can be done with situation, however it is not limited to the execution sequence of above steps.
Further to illustrate this programme, present invention also offers a kind of application of the replacement method of regional area in image
Example, referring to Fig. 7 and Fig. 8, in the application example, benchmark image has particular application as model's image, replaces image concrete application
For user images, the second area and the second area are positive facial image, the application example specifically include as
Lower content：
Referring to Fig. 7, facial feature points detection is carried out to input model's image and user images first and is split with face.It is special
A basis for sign point detection is advanced row Face datection with locating human face's image position, due to this patent mainly for front and
Unobstructed face, presently disclosed a large amount of Face datection algorithms can be achieved accurately to detect.After completing Face datection, to inspection
The face that measures carries out feature point extraction, and characteristic point is commonly referred to as the eyebrow of face, eyes, nose, mouth, the position of face mask
Put, the method for feature point extraction is more including based on deep learning, based on active contour model, and the application is not limited to use
Which kind of method, it is only necessary to ensure that the extraction of characteristic point is accurate.
Using the human face characteristic point extracted, using the priori of human face structure, face foreground and background can be obtained
A primary segmentation.The dividing method of many efficiently and accuratelies can be utilized on the basis of this primary segmentation (such as
GraphCut, grabCut, oneCut, ACM etc.) carry out face segmentation.The mask of human face region can be obtained by above step
With the coordinate of characteristic point.
Then using the human face characteristic point extracted by user images align model's image.Such as：Utilize model's characteristic point
With user characteristics point, and by the use of affine transformation as geometric deformation model, by Least Square Method from user characteristics point to mould
The affine transformation parameter of feature point transformation.And user images are carried out with deformation acquisition using the affine transformation parameter and is schemed with model
Picture size is identical, the user images that face location matches substantially.It is also required to while deformation is carried out to user images with identical
Parameter deformation is carried out to face mask and the feature point coordinates of user.
Subsequently, using the feature point coordinates after user's geometric transformation as benchmark, by the face deformation of model's image come with
The face geometry of user is consistent.At this moment problem is converted to the image precise registration problem with known matching characteristic point.Such as
Fruit now directly uses smart method for registering, such as carries out affine transformation using local triangulation networks or use Spline Interpolation Method
Matched, image can be caused unnatural, such as triangle network method can have a fracture of pixel between triangle gridding, and batten
Interpolation can cause the excessive distortion of image.In order to eliminate above phenomenon, this patent sets a protection outside characteristic point region
Rectangle frame (by way of the profile point increase of rectangle frame is characterized a little), deformation is mainly carried out in rectangle frame, is so had
The scope of the control deformation of effect.Followed by the relative coordinate of characteristic point, more characteristic points are generated.Such as by characteristic point
The triangulation network is built, and Loop subdivision subdivisions are carried out to the triangulation network.A large amount of control points after extension can cause essence to match somebody with somebody
Will definitely be to obtain very smooth natural image.Above deformation process is intended to carry out the image of model and the mask of model's face.
The face of model after deformation finally is replaced using the face of user, the region of replacement is image face after two width deformation
The common factor of mask.And naturalization processing is carried out to the face after replacement so that the color of image is consistent.Naturalization processing can make
With graph cut method or other image interfusion methods.
Referring to Fig. 8, above method is carried out under the assumed condition that two images can replace naturally, but not
It is that any image for including two width faces can carry out nature replacement.In order to improve the detection that the experience of user needs to automate
Whether two images can mutually replace.The condition that can be replaced is that the face of two images is substantially consistent.And cause two width
It is several aspects that image, which replaces unnatural principal element,：A. whether wearing spectacles；B. whether the posture of face is consistent；c.
Whether illumination is consistent.The situation for generally working as user's wearing spectacles is difficult to handle naturally (difference for considering leg of spectacles), especially
It is at ear, is exactly not allow it to pass through verification if user's wearing spectacles in processing procedure thus.
Thus need with eyeglass detection method in method of calibration, can be with due to the position of human face characteristic point can be obtained
Easily obtain eye areas image, then decide whether it is that the models of glasses can determine whether by two classification algorithm training one
Whether wearing spectacles.
And to be also another influence to replace the principal element of effect pose problem, such as when model is that left side hides face and user
It is that general image after then being replaced with front face will be very unnatural.Thus need by the parameter for the posture for calculating face,
And only allow the image with smaller posture difference to by verification, characterize the parameter of head part's posture have 3 be respectively yaw angle,
The angle of pitch and roll angle, and these three parameters can calculate acquisition by the feature point coordinates of face.
Illumination factor is the factor that another mainly influences to change face, facial illumination patterns situation, in skewness and
Difference angle can be estimated when smaller by simple light intensity and compensation is relaxed, and when degree of irregularity is higher, it is few
Amount strength difference compensation after even can cause with being discord around face, in order to avoid the appearance of this phenomenon, first estimate
The light conditions of face, light is not allowed for be distributed highly non-uniform image by verification, and it is only few to different facial zones
The image for measuring light intensity difference carries out illumination compensation, then passes through verification.
It was found from foregoing description, application example of the invention reduce can realize the replacement of regional area in image be
The hardware cost of system, dispose more flexibly and fast, dilatation easy to maintain, there is provided more stable service.More preferable Consumer's Experience, and
The calculating cost of its replacement process is low, and real-time is high, image procossing validity is high.
Embodiments of the invention seven, which provide, can realize in above-mentioned image Overall Steps in the replacement method of regional area
A kind of image in regional area replacement system a kind of embodiment, referring to Fig. 9, regional area in described image
Replacement system specifically includes following content：
Characteristic point acquiring unit 10, for obtaining the first area in benchmark image respectively and replacing the secondth area in image
The characteristic point in domain and the positional information of characteristic point；Wherein, area attribute and institute of the first area in the benchmark image
It is identical to state area attribute of the second area in the benchmark image.
Dividing processing unit 20, for the benchmark image and replace image in the first area and second area
Carry out foreground segmentation process.
Image deformation unit 30 is replaced, for the positional information according to the first area and the characteristic point of second area,
Global deformation process is carried out to the replacement image so that the size for replacing the second area in image and position after deformation with
The size of the first area and the difference of position are in preset range.
Benchmark image deformation unit 40, for being believed according to the position of the first area after deformation and the characteristic point of second area
Breath, partial row's deformation process is entered to the first area in the benchmark image and its neighboring region so that the first area with
The geometry of the second area is identical；Wherein, the neighboring region is the extended area centered on the first area
In and the region adjacent with the first area.
Regional area replacement unit 50, for the first area after deformation to be replaced with to the second area after deformation, obtain
Include the benchmark image of the second area.
It was found from foregoing description, using in the image of the replacement system of regional area in the image in embodiments of the invention
Rapidly and reliably property is high for the replacement processing procedure of regional area, and improves the accuracy of the replacement of regional area, can protect
Regional area after card is replaced does not change, and solves in image regional area deformation and partial zones in regional area replacement technology
It is overseas enclose gross distortion caused by composograph invalidity problem, while reduce the time-consuming and cost for replacing processing.
Embodiments of the invention eight, which provide, can realize in above-mentioned image Overall Steps in the replacement method of regional area
A kind of image in regional area alternative a kind of embodiment, the alternative of regional area in described image
Specifically include following content：
Memory, processor and storage on a memory and the computer program that can run on a processor, the processing
Device realizes the step of any one replacement method when performing the computer program, such as realizes following step：
Step 100：Respectively obtain benchmark image in first area and replace image in second area characteristic point and
The positional information of characteristic point；Wherein, area attribute of the first area in the benchmark image exists with the second area
Area attribute in the benchmark image is identical.
Step 200：Prospect point is carried out to the first area and second area in the benchmark image and replacement image
Cut processing.
Step 300：According to the first area and the positional information of the characteristic point of second area, the replacement image is entered
The global deformation process of row so that the chi of the size for replacing the second area in image and position and the first area after deformation
The very little and difference of position is in preset range.
Step 400：According to the first area after deformation and the positional information of the characteristic point of second area, to the reference map
First area and its neighboring region as in carry out local deformation processing so that the first area is several with the second area
What structure is identical；Wherein, the neighboring region is in extended area centered on the first area and with described first
The adjacent region in region.
Step 500：First area after deformation is replaced with into the second area after deformation, obtains including the second area
Benchmark image.
Embodiments of the invention nine, which provide, can realize in above-mentioned image Overall Steps in the replacement method of regional area
A kind of computer-readable recording medium a kind of embodiment, the computer-readable recording medium specifically includes as follows
Content：
Computer program is stored with computer-readable recording medium, realizes and appoints when the computer program is executed by processor
The step of one replacement method, such as realize following step：
Step 100：Respectively obtain benchmark image in first area and replace image in second area characteristic point and
The positional information of characteristic point；Wherein, area attribute of the first area in the benchmark image exists with the second area
Area attribute in the benchmark image is identical.
Step 200：Prospect point is carried out to the first area and second area in the benchmark image and replacement image
Cut processing.
Step 300：According to the first area and the positional information of the characteristic point of second area, the replacement image is entered
The global deformation process of row so that the chi of the size for replacing the second area in image and position and the first area after deformation
The very little and difference of position is in preset range.
Step 400：According to the first area after deformation and the positional information of the characteristic point of second area, to the reference map
First area and its neighboring region as in carry out local deformation processing so that the first area is several with the second area
What structure is identical；Wherein, the neighboring region is in extended area centered on the first area and with described first
The adjacent region in region.
Step 500：First area after deformation is replaced with into the second area after deformation, obtains including the second area
Benchmark image.
It should also be noted that, herein, such as first and second or the like relational terms are used merely to one
Entity or operation make a distinction with another entity or operation, and not necessarily require or imply between these entities or operation
Any this actual relation or order be present.Moreover, term " comprising ", "comprising" or its any other variant are intended to contain
Lid nonexcludability includes, so that process, method, article or equipment including a series of elements not only will including those
Element, but also the other element including being not expressly set out, or it is this process, method, article or equipment also to include
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Other identical element also be present in process, method, article or equipment including the key element.
Above example is merely to illustrate technical scheme, rather than its limitations；Although with reference to the foregoing embodiments
The present invention is described in detail, it will be understood by those within the art that：It still can be to foregoing each implementation
Technical scheme described in example is modified, or carries out equivalent substitution to which part technical characteristic；And these are changed or replaced
Change, the essence of appropriate technical solution is departed from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (10)
1. the replacement method of regional area in a kind of image, it is characterised in that the replacement method includes：
The first area in benchmark image is obtained respectively and replaces the characteristic point of second area in image and the position of characteristic point
Information；Wherein, area attribute of the first area in the benchmark image and the second area are in the benchmark image
In area attribute it is identical；
Foreground segmentation process is carried out to the first area and second area respectively in the benchmark image and replacement image；
According to the first area and the positional information of the characteristic point of second area, the replacement image is carried out at global deformation
Reason so that the size for replacing the second area in image and position and the size of the first area and the difference of position after deformation
Value is in preset range；
According to the first area after deformation and the positional information of the characteristic point of second area, to the firstth area in the benchmark image
Domain and its neighboring region carry out local deformation processing so that the first area is identical with the geometry of the second area；
Wherein, the neighboring region is in the extended area centered on the first area and adjacent with the first area
Region；
And the first area after deformation is replaced with into the second area after deformation, obtain the benchmark for including the second area
Image.
2. replacement method according to claim 1, it is characterised in that the first area obtained respectively in benchmark image
With replace image in the characteristic point of second area and the positional information of characteristic point, including：
Respectively to the benchmark image and replace image carry out target area detection of attribute, and extract respectively the first area and
Each characteristic point of second area and the positional information of each characteristic point.
3. replacement method according to claim 1, it is characterised in that the firstth area in the benchmark image of acquisition respectively
After the characteristic point of second area and the positional information of characteristic point in domain and replacement image, methods described also includes：
According to the first area and the characteristic point of second area and the positional information of characteristic point in image are replaced, described in judgement
Whether the attribute information for being used to examine of first area is with the attribute information for being used to examine in corresponding second area default
Range of attributes in, that is, judge it is described replacement image whether pass through inspection；
If so, then the first area and second area are carried out at foreground segmentation in the benchmark image and replacement image
Reason；
Otherwise, it is determined that the second area currently replaced in image can not replace the first area in benchmark image, and output can not
That replaces informs prompting.
4. the replacement method according to claim 1 or 3, it is characterised in that described in the benchmark image and replacement image
In foreground segmentation process is carried out to the first area and second area, including：
According to the structural relation between each characteristic point of the first area, by first area and environment in the benchmark image
Background is split, and obtains the mask of the first area and the first area；
And according to the structural relation between each characteristic point of the second area, by second area in the replacement image
Split with environmental background, obtain the mask of the second area and the second area.
5. replacement method according to claim 1, it is characterised in that described according to the first area and second area
The positional information of characteristic point, global deformation process is carried out to the replacement image so that replace in image second after deformation
The size in region and position are in preset range with the size of the first area and the difference of position, including：
According to the first area and the characteristic point of second area, geometric deformation model is established using global distortion removal method；
Global deformation process is carried out to the replacement image based on the geometric deformation model；
And the deformation results according to the replacement image, the positional information of the corresponding characteristic point for adjusting the second area and
The mask of the second area.
6. replacement method according to claim 1, it is characterised in that in the first area according to after deformation and second
The positional information of the characteristic point in region, the first area in the benchmark image and its neighboring region are carried out at local deformation
Reason so that before the first area is identical with the geometry of the second area, in addition to：
The position relationship network of the characteristic point of first area is built according to the positional information of N number of characteristic point of first area；And
The position relationship network of the characteristic point of second area is built according to the positional information of N number of characteristic point of second area；
And the position relationship network of the first area is finely divided, obtain M characteristic point so that the first area
Characteristic point sum be M+N；And the position relationship network of the second area is finely divided, M characteristic point is obtained,
So that the characteristic point sum of the second area is also M+N；
Wherein, M and N is positive integer.
7. replacement method according to claim 1, it is characterised in that the first area and the secondth area according to after deformation
The positional information of the characteristic point in domain, local deformation processing is carried out to the first area in the benchmark image and its neighboring region,
So that the first area is identical with the geometry of the second area, including：
Set respectively outside first area and each characteristic point for replacing the second area in image in the benchmark image
Protecting frame, wherein, the position of the protecting frame is built according to the central point of protecting frame, wherein, the central point of protecting frame is by complete
The center in the closing convex closure region of the circumference composition of portion's characteristic point；
And according to the first area after deformation and the positional information of the characteristic point of second area, in the benchmark image
In protecting frame outside each characteristic point of first area, the carry out deformation process of the first human face region controlled each characteristic point,
So that the first area is identical with the geometry of the second area.
8. replacement method according to claim 1, it is characterised in that the first area by after deformation replaces with deformation
Second area afterwards, the benchmark image for including the second area is obtained, including：
First area after deformation is replaced with into the second area after deformation, and the region replaced is the first area after deformation
The intersection area of the mask of second area after mask and deformation, obtain the benchmark image for including the second area；
And naturalization fusion treatment is carried out to the second area in the benchmark image including the second area.
9. the alternative of regional area in a kind of image, including memory, processor and storage on a memory and can located
The computer program run on reason device, it is characterised in that realize that right such as will described in the computing device during computer program
The step of seeking any one of 1 to 8 methods described.
10. computer-readable recording medium, it is stored thereon with computer program, it is characterised in that the computer program is processed
Realized when device performs such as the step of any one of claim 1 to 8 methods described.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710706278.4A CN107507216B (en) | 2017-08-17 | 2017-08-17 | Method and device for replacing local area in image and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710706278.4A CN107507216B (en) | 2017-08-17 | 2017-08-17 | Method and device for replacing local area in image and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107507216A true CN107507216A (en) | 2017-12-22 |
| CN107507216B CN107507216B (en) | 2020-06-09 |
Family
ID=60692089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710706278.4A Expired - Fee Related CN107507216B (en) | 2017-08-17 | 2017-08-17 | Method and device for replacing local area in image and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107507216B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154132A (en) * | 2018-01-10 | 2018-06-12 | 马上消费金融股份有限公司 | A kind of identity card text extraction method, system and equipment and storage medium |
| CN108765265A (en) * | 2018-05-21 | 2018-11-06 | 北京微播视界科技有限公司 | Image processing method, device, terminal device and storage medium |
| CN109344724A (en) * | 2018-09-05 | 2019-02-15 | 深圳伯奇科技有限公司 | A kind of certificate photo automatic background replacement method, system and server |
| CN109978754A (en) * | 2017-12-28 | 2019-07-05 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN110290395A (en) * | 2019-06-14 | 2019-09-27 | 北京奇艺世纪科技有限公司 | A kind of image processing method, device and computer readable storage medium |
| CN110298826A (en) * | 2019-06-18 | 2019-10-01 | 合肥联宝信息技术有限公司 | A kind of image processing method and device |
| WO2020019904A1 (en) * | 2018-07-23 | 2020-01-30 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, and computer device and storage medium |
| CN111447389A (en) * | 2020-04-22 | 2020-07-24 | 广州酷狗计算机科技有限公司 | Video generation method, device, terminal and storage medium |
| CN111768468A (en) * | 2020-06-30 | 2020-10-13 | 北京百度网讯科技有限公司 | Image filling method, device, equipment and storage medium |
| WO2021031506A1 (en) * | 2019-08-22 | 2021-02-25 | 北京市商汤科技开发有限公司 | Image processing method and apparatus, electronic device, and storage medium |
| TWI725746B (en) * | 2019-02-26 | 2021-04-21 | 大陸商騰訊科技（深圳）有限公司 | Image fusion method, model training method, and related device |
| CN112990123A (en) * | 2021-04-26 | 2021-06-18 | 北京世纪好未来教育科技有限公司 | Image processing method, apparatus, computer device and medium |
| CN113256499A (en) * | 2021-07-01 | 2021-08-13 | 北京世纪好未来教育科技有限公司 | Image splicing method, device and system |
| WO2022002032A1 (en) * | 2020-06-29 | 2022-01-06 | 北京灵汐科技有限公司 | Image-driven model training and image generation |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103873741A (en) * | 2014-04-02 | 2014-06-18 | 北京奇艺世纪科技有限公司 | Method and device for substituting area of interest in video |
| CN104657974A (en) * | 2013-11-25 | 2015-05-27 | 腾讯科技（上海）有限公司 | Image processing method and device |
| CN105069746A (en) * | 2015-08-23 | 2015-11-18 | 杭州欣禾圣世科技有限公司 | Video real-time human face substitution method and system based on partial affine and color transfer technology |
| CN105118082A (en) * | 2015-07-30 | 2015-12-02 | 科大讯飞股份有限公司 | Personalized video generation method and system |
| CN106204435A (en) * | 2016-06-27 | 2016-12-07 | 北京小米移动软件有限公司 | Image processing method and device |
| KR20170050980A (en) * | 2015-11-02 | 2017-05-11 | 주식회사 파이브지티 | Face recognition system and method of adapting to environment |
| CN106686463A (en) * | 2016-12-09 | 2017-05-17 | 天脉聚源(北京)传媒科技有限公司 | Video role replacing method and apparatus |
-
2017
- 2017-08-17 CN CN201710706278.4A patent/CN107507216B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657974A (en) * | 2013-11-25 | 2015-05-27 | 腾讯科技（上海）有限公司 | Image processing method and device |
| CN103873741A (en) * | 2014-04-02 | 2014-06-18 | 北京奇艺世纪科技有限公司 | Method and device for substituting area of interest in video |
| CN105118082A (en) * | 2015-07-30 | 2015-12-02 | 科大讯飞股份有限公司 | Personalized video generation method and system |
| CN105069746A (en) * | 2015-08-23 | 2015-11-18 | 杭州欣禾圣世科技有限公司 | Video real-time human face substitution method and system based on partial affine and color transfer technology |
| KR20170050980A (en) * | 2015-11-02 | 2017-05-11 | 주식회사 파이브지티 | Face recognition system and method of adapting to environment |
| CN106204435A (en) * | 2016-06-27 | 2016-12-07 | 北京小米移动软件有限公司 | Image processing method and device |
| CN106686463A (en) * | 2016-12-09 | 2017-05-17 | 天脉聚源(北京)传媒科技有限公司 | Video role replacing method and apparatus |
Non-Patent Citations (2)
| Title |
|---|
| YUAN LIN ET AL.: "Face replacement with large-pose differences", 《PROCEEDING OF THE 20TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA》 * |
| 高岩 等: "约束条件下的人脸五官替换算法", 《中国图象图形学报》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109978754A (en) * | 2017-12-28 | 2019-07-05 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN108154132A (en) * | 2018-01-10 | 2018-06-12 | 马上消费金融股份有限公司 | A kind of identity card text extraction method, system and equipment and storage medium |
| CN108765265A (en) * | 2018-05-21 | 2018-11-06 | 北京微播视界科技有限公司 | Image processing method, device, terminal device and storage medium |
| WO2020019904A1 (en) * | 2018-07-23 | 2020-01-30 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, and computer device and storage medium |
| US11455729B2 (en) | 2018-07-23 | 2022-09-27 | Tencent Technology (Shenzhen) Company Limited | Image processing method and apparatus, and storage medium |
| JP7052079B2 (en) | 2018-07-23 | 2022-04-11 | テンセント・テクノロジー・（シェンジェン）・カンパニー・リミテッド | Image processing methods, devices, computer devices and computer programs |
| JP2021519995A (en) * | 2018-07-23 | 2021-08-12 | テンセント・テクノロジー・（シェンジェン）・カンパニー・リミテッド | Image processing methods, devices, computer devices and computer programs |
| CN109344724A (en) * | 2018-09-05 | 2019-02-15 | 深圳伯奇科技有限公司 | A kind of certificate photo automatic background replacement method, system and server |
| TWI725746B (en) * | 2019-02-26 | 2021-04-21 | 大陸商騰訊科技（深圳）有限公司 | Image fusion method, model training method, and related device |
| CN110290395B (en) * | 2019-06-14 | 2021-05-25 | 北京奇艺世纪科技有限公司 | Image processing method and device and computer readable storage medium |
| CN110290395A (en) * | 2019-06-14 | 2019-09-27 | 北京奇艺世纪科技有限公司 | A kind of image processing method, device and computer readable storage medium |
| CN110298826A (en) * | 2019-06-18 | 2019-10-01 | 合肥联宝信息技术有限公司 | A kind of image processing method and device |
| WO2021031506A1 (en) * | 2019-08-22 | 2021-02-25 | 北京市商汤科技开发有限公司 | Image processing method and apparatus, electronic device, and storage medium |
| CN111447389A (en) * | 2020-04-22 | 2020-07-24 | 广州酷狗计算机科技有限公司 | Video generation method, device, terminal and storage medium |
| CN111447389B (en) * | 2020-04-22 | 2022-11-04 | 广州酷狗计算机科技有限公司 | Video generation method, device, terminal and storage medium |
| WO2022002032A1 (en) * | 2020-06-29 | 2022-01-06 | 北京灵汐科技有限公司 | Image-driven model training and image generation |
| CN111768468A (en) * | 2020-06-30 | 2020-10-13 | 北京百度网讯科技有限公司 | Image filling method, device, equipment and storage medium |
| CN111768468B (en) * | 2020-06-30 | 2023-08-04 | 北京百度网讯科技有限公司 | Image filling method, device, equipment and storage medium |
| CN112990123A (en) * | 2021-04-26 | 2021-06-18 | 北京世纪好未来教育科技有限公司 | Image processing method, apparatus, computer device and medium |
| CN113256499A (en) * | 2021-07-01 | 2021-08-13 | 北京世纪好未来教育科技有限公司 | Image splicing method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107507216B (en) | 2020-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107507216A (en) | The replacement method of regional area, device and storage medium in image | |
| CN107564080A (en) | A kind of replacement system of facial image | |
| CN107507217A (en) | Preparation method, device and the storage medium of certificate photo | |
| CN107610202A (en) | Marketing method, equipment and the storage medium replaced based on facial image | |
| CN109697688B (en) | Method and device for image processing | |
| CN105354876B (en) | A kind of real-time volume fitting method based on mobile terminal | |
| JP4785880B2 (en) | System and method for 3D object recognition | |
| CN110096925A (en) | Enhancement Method, acquisition methods and the device of Facial Expression Image | |
| CN109801380A (en) | A kind of method, apparatus of virtual fitting, storage medium and computer equipment | |
| CN109952594A (en) | Image processing method, device, terminal and storage medium | |
| CN103839223A (en) | Image processing method and image processing device | |
| CN106652015B (en) | Virtual character head portrait generation method and device | |
| CN108305312A (en) | The generation method and device of 3D virtual images | |
| US11562536B2 (en) | Methods and systems for personalized 3D head model deformation | |
| CN105956997A (en) | Image deformation treatment method and device | |
| US11587288B2 (en) | Methods and systems for constructing facial position map | |
| CN110288715B (en) | Virtual necklace try-on method and device, electronic equipment and storage medium | |
| CN109886144B (en) | Virtual trial sending method and device, computer equipment and storage medium | |
| CN109410119A (en) | Mask image distortion method and its system | |
| CN104809687A (en) | Three-dimensional human face image generation method and system | |
| US11461970B1 (en) | Methods and systems for extracting color from facial image | |
| CN107145224A (en) | Human eye sight tracking and device based on three-dimensional sphere Taylor expansion | |
| CN110287894A (en) | A kind of gesture identification method and system for ultra-wide angle video | |
| WO2022197430A1 (en) | Methods and systems for forming personalized 3d head and facial models | |
| CN113516755B (en) | Image processing method, image processing apparatus, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200609 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |