CN108961299A - A kind of foreground image preparation method and device - Google Patents

A kind of foreground image preparation method and device Download PDF

Info

Publication number
CN108961299A
CN108961299A CN201710351704.7A CN201710351704A CN108961299A CN 108961299 A CN108961299 A CN 108961299A CN 201710351704 A CN201710351704 A CN 201710351704A CN 108961299 A CN108961299 A CN 108961299A
Authority
CN
China
Prior art keywords
value
pixel
video frame
target video
rgb
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710351704.7A
Other languages
Chinese (zh)
Other versions
CN108961299B (en
Inventor
王明琛
梅元刚
刘鹏
陈宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kingsoft Cloud Network Technology Co Ltd
Beijing Kingsoft Cloud Technology Co Ltd
Original Assignee
Beijing Kingsoft Cloud Network Technology Co Ltd
Beijing Kingsoft Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kingsoft Cloud Network Technology Co Ltd, Beijing Kingsoft Cloud Technology Co Ltd filed Critical Beijing Kingsoft Cloud Network Technology Co Ltd
Priority to CN201710351704.7A priority Critical patent/CN108961299B/en
Publication of CN108961299A publication Critical patent/CN108961299A/en
Application granted granted Critical
Publication of CN108961299B publication Critical patent/CN108961299B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Processing Of Color Television Signals (AREA)

Abstract

The embodiment of the invention provides a kind of foreground image preparation method and device, method includes: to obtain target video frame；Wherein, the target video frame is any frame image in original video；According to the first rgb value of each pixel of the target video frame, the curtain type of the background image of second rgb value and the target video frame of each pixel in the background image of the target video frame is determined；According to the first rgb value and the second rgb value of each pixel, picture value calculation formula is covered according to corresponding with the curtain type, obtain each pixel covers picture value；Picture value is covered according to each pixel, third rgb value of each pixel in the foreground image of the target video frame is determined, obtains the foreground image of the target video frame.Using the embodiment of the present invention, it is possible to reduce color spillover.

Description

A kind of foreground image preparation method and device
Technical field
The present invention relates to technical field of video processing, more particularly to a kind of foreground image preparation method and device.
Background technique
One video frame can be regarded as the composograph obtained after a width is synthesized by foreground image and background image.Video The problem of background replacement of frame is studied be by video frame foreground image and background image separate, and by isolated prospect Image is synthesized in an other background image, including two big steps: being scratched picture and synthesis, is scratched as (also referred to as scratching figure) is by video frame In the process that extracts of foreground image, synthesis is then that the foreground image that will be extracted is placed on shape in new background image At a new video frame.Scratch as and synthesis be that special video effect makes essential means, this technology can by performer or Person host, main broadcaster etc., which are embedded into, realizes certain program effect in virtual environment.Because of green and blue and human body complexion It differs greatly, easier can carry out scratching figure, therefore use pure green or pure blue when shooting video under normal conditions Curtain as background.
The example of common background replacement is weather forecast in daily life.When we see TV, it appears that day Gas forecaster is station before a width weather nephogram, but in fact, weatherman is station bat in the casting made above of blue curtain It takes the photograph to obtain original video frame, then weatherman is plucked out from original video frame by software for editing and is superimposed is synthesized to Obtain new video frame on weather nephogram, that is, background image replaced with into weather nephogram by blue curtain, thus produce from The effect watched on TV.
Scratching picture and synthetic technology can be stated with synthesis equation, and synthesis equation is as follows:
C=α F+ (1- α) B
Wherein, C, F and B respectively indicate composograph, foreground image and background image, and each pixel is pointed out in composograph Color value be formed by stacking by the corresponding color value of foreground image and the corresponding color value of background image.α is known as covering picture, each picture α value at vegetarian refreshments indicates the percentage of foreground color in the color value of corresponding pixel points in composograph C, or indicates the pixel The opacity of point, the range of α is [0,1].
In view of above-mentioned synthesis equation, in RGB color, to each pixel in video frame, respectively in R, G, B 3 1 equation is established on a channel, the equation group of composition is as follows:
When composograph C is gray level image, 1 equation is corresponding with to each pixel in C, 3 unknown quantitys F, B and α.When composograph C is color image, each pixel in C is then corresponding with 3 equations and 7 unknown numbers, removes above-mentioned side C in journey groupR,CG,CBIn addition, remaining is unknown quantity, it is seen that is the problem of can not accurately solving in Kou Tu question essence.
By analysis above it can be found that the committed step of background replacement is stingy figure i.e. acquisition foreground image, that is, Find out F, B and the α in composograph at each pixel.For the stingy drawing method under green curtain/indigo plant curtain background, since background is pure Green or pure blue, the picture value α that covers at the pixel of foreground image marginal portion are affected by background color, are caused Picture value of covering at the pixel for the foreground image marginal portion being calculated when scratching figure differs larger with actual value, so that scratching , that is, there is color spillover in the marginal portion residual blue or green pixel of foreground image out.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of foreground image preparation method and device, with reduce color overflow it is existing As.Specific technical solution is as follows:
In order to achieve the above objectives, the embodiment of the invention discloses a kind of foreground image preparation methods, which comprises
Obtain target video frame；Wherein, the target video frame is any frame image in original video；
According to the first rgb value of each pixel of the target video frame, determine that each pixel is regarded in the target The curtain type of the background image of the second rgb value and the target video frame in the background image of frequency frame；
According to the first rgb value and the second rgb value of each pixel, picture value meter is covered according to corresponding with the curtain type Formula is calculated, obtain each pixel covers picture value；
Picture value is covered according to each pixel, determines of each pixel in the foreground image of the target video frame Three rgb values obtain the foreground image of the target video frame.
In order to achieve the above objectives, the embodiment of the invention also discloses a kind of foreground images to obtain device, and described device includes:
Module is obtained, for obtaining target video frame；Wherein, the target video frame is any frame figure in original video Picture；
First determining module determines each picture for the first rgb value according to each pixel of the target video frame The curtain of the background image of second rgb value and the target video frame of the vegetarian refreshments in the background image of the target video frame Cloth type；
First obtains module, for the first rgb value and the second rgb value according to each pixel, according to the curtain Type is corresponding to cover picture value calculation formula, and obtain each pixel covers picture value；
Second determining module determines each pixel in the target video for covering picture value according to each pixel Third rgb value in the foreground image of frame obtains the foreground image of the target video frame.
Foreground image preparation method and device provided in this embodiment, according to the curtain class of the background image of target video frame Type obtains the picture value of covering of each pixel, Ke Yiyou targetedly according to picture value calculation formula is covered corresponding to curtain type Effect reduces background color to the influence of picture value is covered, to reduce color spillover, reaches preferably stingy figure effect.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.
Fig. 1 is a kind of flow diagram of foreground image preparation method provided in an embodiment of the present invention；
(a) is the original image of a video frame in Fig. 2, is (b) the corresponding guidance figure of the video frame, (c) is video frame pair The input picture answered (d) is the corresponding output image of the video frame；
(a) indicates guidance figure G in the neighborhood w of pixel k in Fig. 3kInterior value (b) indicates input picture P in pixel k Neighborhood wkInterior value (c) indicates GP in the neighborhood w of pixel kkInterior value (d) is G2In the neighborhood w of pixel kk's Value；
Fig. 4 is the functional arrangement for the calculation formula that picture value is covered in adjustment in a specific embodiment provided in an embodiment of the present invention Picture；
(a), (b) are two group searching directions in a specific embodiment provided in an embodiment of the present invention in Fig. 5；
(a), (b) are respectively (a) in Fig. 5, the corresponding search order of the direction of search shown in (b) in Fig. 6；
Fig. 7 is the process flow diagram of a specific embodiment provided in an embodiment of the present invention；
Fig. 8 is the effect picture in one provided in an embodiment of the present invention experiment；
Fig. 9 is original image corresponding to experiment effect figure shown in Fig. 8；
Figure 10 is the structural schematic diagram that a kind of foreground image provided in an embodiment of the present invention obtains device.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.
To solve prior art problem, the embodiment of the invention provides a kind of foreground image preparation method and devices.Below A kind of foreground image preparation method is provided for the embodiments of the invention first to be described in detail.
It should be noted that a kind of executing subject of foreground image preparation method provided by the present embodiment can be one kind Video coding apparatus, wherein the video coding apparatus can be the plug-in unit in existing Video coding software, alternatively, independent function Energy software, is such as broadcast live software, this is all reasonable.Also, the video coding apparatus can be applied in terminal, can also apply In server.
Fig. 1 is a kind of flow diagram of foreground image preparation method provided in an embodiment of the present invention, this method comprises:
S101 obtains target video frame；Wherein, target video frame is any frame image in original video；
It should be understood that video frame can be regarded as the composite diagram obtained after a width is synthesized by foreground image and background image Picture, usual foreground image is interested target object, background image is environment locating for the target object, for example, personage stands A width video frame by the sea, foreground image are personage, and background image is the environment on seashore.For green curtain or blue curtain video, It is shot under the background of green curtain or blue curtain, and therefore, the background image of each video frame is the green of pure color Curtain or blue curtain, foreground image are the target objects such as captured personage.
S102 determines each pixel in target video frame according to the first rgb value of each pixel of target video frame Background image in the second rgb value and target video frame background image curtain type.
Wherein, curtain type can be green curtain, blue curtain etc..The rgb value of pixel is that pixel is red in RGB color The value of turquoise three components, wherein R indicates that red component, G indicate that green component, B indicate blue component.First rgb value CB, CG,CRFor the rgb value of pixel in target video frame, the second rgb value BB,BG,BRFor RGB of each pixel in background image It is worth, the third rgb value F in step S104B,FG,FRFor rgb value of each pixel in foreground image.
In practical application, the first rgb value of above-mentioned each pixel according to target video frame determines that each pixel exists The step of the second rgb value in the background image of target video frame, may include:
According to the first rgb value of each pixel of target video frame, the tone H component value of each pixel is obtained；Its In, value determined by the first rgb value of the pixel according to the tone H component value of any pixel point；
According to the tone H component value of each pixel, determine each pixel in the background image of target video frame Second rgb value.
Specifically, the above-mentioned tone H component value according to each pixel, determines each pixel in the back of target video frame The step of the second rgb value in scape image, comprising:
The number for counting pixel corresponding to each tone H component value, by the largest number of tone H components of pixel It is worth the tone H component value of the background image as target video frame；
According to the tone H component value of the background image of target video frame, judge target video frame background image whether be Green curtain or blue curtain；
In the case where the background image of target video frame is green curtain, by the first of the first kind pixel of target video frame The average value of rgb value is determined as second rgb value of each pixel in the background image of target video frame, wherein the first kind Pixel is pixel of the absolute value of the difference less than the first preset threshold of tone H component value tone value corresponding with green；
In the case where the background image of target video frame is blue curtain, by the first of the second class pixel of target video frame The average value of rgb value is determined as second rgb value of each pixel in the background image of target video frame, wherein the second class Pixel is pixel of the absolute value of the difference less than the second preset threshold of tone H component value tone value corresponding with blue.
For example, target video frame is transformed into hsv color space from RGB color, HSV is a kind of very intuitive face The colour space, the parameter of color is respectively in this color space: tone (H), saturation degree (S), lightness (V).Tone H differences in angle Amount, value range 0-360 are calculated counterclockwise since red, and red is 0, green 120, blue 360；It is full Color is indicated close to the degree of spectrum colour with degree S, and usual value range is 0%-100%, and the bigger expression color of value is more saturated； The usual value range of lightness V is 0% (black) -100% (white).Here estimated using hsv color space background color be because HSV is a kind of intuitive color model for users, and H component can be very good description colouring information, when a pixel is corresponding Tone H component value value can be determined that the pixel is green when being 120 or so, when tone H component value value is 240 or so When can be determined that the pixel for blue.Image is as follows by the conversion formula in RGB color to hsv color space, wherein R, G, B respectively indicate a pixel tri- component values of R, G, B, H, S, V in RGB color in image and respectively indicate the pixel The value in tri- channels H, S, V in hsv color space:
V=max (R, G, B)
If H < 0then H=H+360
Furthermore it is also possible to which target video frame is transformed into other color spaces by RGB color to obtain each picture The tone H component value of vegetarian refreshments is not done herein such as the method that HSL color space, specific conversion process are referred to the prior art It repeats.
After target video frame is transformed into hsv color space from RGB color, then count all of target video frame Histogram of the pixel on tone H component, as previously described, in 0-360, statistic histogram refers to the value range of tone H Value number of the tone H component value of all pixels point of target video frame in each value of 0-360 is counted, then takes number most Tone H component value of more that tone H components as background image, for example, tone H component value be 120 pixel number At most, then can think that the tone H component value of background image is 120.It is reasonable for doing so, because for background image For green curtain or the composograph of blue curtain, green pixel or blue pixel point account for all pixels point in image ratio it is higher and The value of tone H component value is concentrated very much, the most tone H component value of value can be estimated as background image using this characteristic Tone H component value.
After obtaining the tone H component value of background image, it can be determined that background image is blue curtain or green curtain, is specifically pressed Judged according to following manner:
Judge the exhausted of the tone H component value of the background image of target video frame and the difference of the corresponding tone H component value of green To value whether less than the first preset threshold, if so, indicating that the background image of target video frame image is green curtain；
Otherwise, judge the tone H component value of the background image of target video frame image tone H component value corresponding with blue Absolute value of the difference whether less than the second preset threshold, if so, indicating that the background image of target video frame image is blue curtain.
For example, the corresponding tone H component value of green can with value for 120, if | HB- 120 | < th1, then it represents that Background As being green curtain；Wherein, HBIndicate that the tone H component value of background image, th1 indicate the first preset threshold；
The corresponding tone H component value of blue can with value for 240, if | HB- 240 | < th2, then it represents that background image is Blue curtain；Wherein, HBIndicate that the tone H component value of background image, th2 indicate the first preset threshold.
It should be noted that whether the background image for judging target video frame is green curtain or blue curtain, can be regarded according to target The tone H component value of the background image of frequency frame judges, can also be judged according to other judgment criterias, such as target video frame Color of the color as background image corresponding to middle the largest number of first rgb values of pixel, then according to background image Color judges whether background image is that green curtain or blue curtain, the present embodiment do not limit this.
First preset threshold can be the same or different with the second preset threshold, all be reasonable.In a kind of preferred reality Apply in example th1, th2 can identical and value be 40.
It should be noted that video frame of the present embodiment only for background image for green curtain or blue curtain obtains foreground image, If it is judged that the background image of target video frame then terminates the processing stream to target video frame neither blue curtain is also not green curtain Journey.
If it is judged that background image is green curtain, then the first kind pixel in target video frame is taken, is then calculated separately First rgb value of first kind pixel tri- components of R, G, B average value, as each pixel the second of background image Rgb value BB,BG,BR；
If it is judged that background image is blue curtain, then the second class pixel in target video frame is taken, is then calculated separately First rgb value of the second class pixel tri- components of R, G, B average value, as each pixel the second of background image Rgb value BB,BG,BR。
As it can be seen that the Background color information of target video frame is detected in the present embodiment automatically using hsv color space, without appointing What man-machine interactively all detects background color to each frame of original video automatically, therefore even if certain video frames background light According to dynamic change is influenced, good stingy figure effect can also be obtained.
S103 covers picture value according to corresponding with curtain type according to the first rgb value and the second rgb value of each pixel Calculation formula, obtain each pixel covers picture value.
After obtaining the information of background image of target video frame, need to obtain each pixel in next step covers picture value.
In practical application, in order to make to obtain to cover picture value more accurate, can be that green curtain or blue curtain make for background image Picture value is covered with different modes to obtain.It is above-mentioned according to each pixel specifically, be green curtain or Lan Mushi in curtain type First rgb value and the second rgb value cover picture value calculation formula according to corresponding with the curtain type, obtain each pixel The step of covering picture value may include:
In the case where the background image of target video frame is green curtain, the mesh is calculated according to following picture value calculation formula of covering Each pixel covers picture value in mark video frame:
In the case where the background image of target video frame is blue curtain, target view is calculated according to following picture value calculation formula of covering Each pixel covers picture value in frequency frame:
Wherein, α indicates that pixel covers picture value, C in target video frameB,CG,CRRespectively indicate the first RGB of the pixel B, G of value, R component value, BB,BG,BRRespectively indicate second rgb value of the pixel in the background image of target video frame B, G, R component value.
For green curtain background image, BG>BB,BG>BR, for blue curtain background image, BB>BG, therefore above-mentioned two cover picture value The denominator of calculation formula is not zero.It should be understood that being two kinds of situations of green curtain or blue curtain for background image, different meters is used It calculates formula calculating and covers picture value, calculated result is more accurate, and calculation amount is smaller, can be handled in real time video, therefore Scheme provided in this embodiment can be applied in live scene.
In practical application, identical processing can also be carried out for green curtain and blue curtain to obtain each pixel covers picture Then value refines the picture value of covering of estimation for example, each pixel of first rough estimate covers picture value, obtains finer Cover picture value.Specifically, above-mentioned the first rgb value and the second rgb value according to each pixel, obtain each pixel covers picture value The step of, may include:
According to the first rgb value and the second rgb value of each pixel, obtain each pixel initially covers picture value, wherein Any pixel point initially covers value determined by the first rgb value and the second rgb value of the pixel according to picture value；
Using guiding figure filtering technique, input picture is filtered using guidance figure to obtain output image, according to output Each pixel covers picture value in image acquisition target video frame, wherein input picture is according to pixel in target video frame Initially cover picture value determine, guidance figure be according in target video frame pixel gray value determination, any pixel point According to gray value determined by the first rgb value of the pixel；
In practical application, it can be directed to each pixel, according to the first rgb value and the second rgb value of the pixel, obtained The difference value of the rgb value of target video frame and the rgb value of background image at the pixel, and according to difference value, obtain target The pixel initially covers picture value in video frame images.
Specifically, can according to the first rgb value and the second rgb value absolute value of the difference calculate difference value, in one kind In preferable implementation, the rgb value of target video frame image and the RGB of background image can be calculated according to following calculation formula The difference value d being worth at the pixel:
D=(CR-BR)2+(CG-BG)2+(CB-BB)2
Wherein, CB,CG,CRRespectively indicate B, G, the R component value of the first rgb value of the pixel in target video frame, BB, BG,BRRespectively indicate B, G, the R component value of second rgb value of the pixel in the background image of target video frame.
It should be understood that difference value the first rgb value of smaller expression and the second rgb value are relatively, i.e., the pixel is back A possibility that scape image, is larger, and the larger difference for indicating the first rgb value and the second rgb value of difference value is larger, then the pixel is A possibility that foreground image, is larger, therefore, can according to following calculation formula calculate target video frame in the pixel it is initial Cover picture value α 1:
Wherein, th2,th2Respectively third predetermined threshold value and the 4th preset threshold, d are difference value.
In the present embodiment, third predetermined threshold value th1400 can be taken, the 4th preset threshold th23600 can be taken, certainly this two A preset threshold can also rule of thumb or actual demand is set as other numerical value, and the present embodiment does not limit this.
Obtain each pixel initially cover picture value after, then guiding figure filtering is carried out to it, to obtain fine covering picture Value.Specifically, can calculate each pixel in target video frame according to following calculation formula covers picture value:
αk=Qk/255
Qk=akGk+bk
Wherein, αkIndicate that pixel k's covers picture value, QkIndicate pixel k corresponding value, G in output image QkFor pixel Point k corresponding value, w in guidance figure GkIndicate the neighborhood formed centered on pixel k, by the pixel of preset quantity, | wk| Indicate neighborhood wkThe number of interior pixel, GiIndicate neighborhood wkInterior ith pixel point corresponding value, P in guidance figure GiIt indicates Neighborhood wkInterior ith pixel point corresponding value in input picture P, ε is preset constant, ak,bkFor variable.
It will be appreciated by persons skilled in the art that needing first to convert target video frame before carrying out guiding figure filtering For gray level image, such as target video frame can be transformed into YCbCr color space by RGB color, wherein Y indicates bright Degree, that is, gray value；And what Cb and Cr was indicated is coloration, is respectively intended to description colors of image and saturation degree.By image from RGB Color space conversion is as follows to the transformational relation to YcbCr color space, wherein R, G, and B respectively indicates each pixel in image Tri- component values of R, G, B in RGB color, Y, Cb, Cr respectively indicate in image each pixel in YCbCr color space The value in middle tri- channels Y, Cb, Cr:
It will be appreciated by persons skilled in the art that the data in the channel Y is only taken to can be obtained by the corresponding ash of color image Degree figure.
The filtering of guiding figure is a kind of image filtering technology, schemes G by a guidance, is filtered to input picture P, So that last output image is generally similar to input picture P, and texture part is similar to guidance figure G.Assuming that output image Can be described with formula are as follows: min for Q in order to enable input picture P and output image Q are as similar as possible | Q-P |2(1)；In order to Keep the texture for exporting image Q and guidance figure G as similar as possible, can be described with formula are as follows:For equation (2), both sides peer-to-peer takes definite integral, to obtain formula: Q=aG+b (3).
In the present embodiment, scheme the corresponding grayscale image of target video frame as guidance, each pixel is initially covered into picture Value is used as input picture P, carries out guiding figure filtering to input picture P by guidance figure G, covers picture value with what is refined, be Facilitate display and calculate, the present embodiment will initially cover picture value α 1 multiplied by the value after 255 as the gray value of input picture P.Show Example property, referring to fig. 2, to clearly show that video frame, (a) shows the original image of a video frame；(c) it indicates initially to cover picture Value, i.e., the input picture P for initially covering picture value and determining of pixel, pays attention to for the ease of showing this in the video frame according to (a) In initially cover picture value be α 1 × 255；(b) the corresponding grayscale image of video frame shown in (a) is indicated, i.e., according to pixel in the video frame The guidance figure G that the gray value of point determines.Guidance figure G and input picture P is single channel image.Formula (3) is a part Linear model, therefore two coefficients a, b are variable related with position in fact.In order to determine a, the value of b considers a wicket wk, so that the pixel in the window meets above formula (1) and formula (2) simultaneously, formula (3) formula can be brought into formula (1), while be in order to prevent calculated to cover picture value excessive, add a penalty term in formula (1), obtained formula are as follows:
To two parameter a in formula (4)k,bkPartial derivative is asked to obtain respectively:
And then it can solve:
Neighborhood w in the present embodimentkRadius can be 20, i.e., centered on pixel k, up and down respectively extend 20 pictures The square area of vegetarian refreshments, i.e., 41 × 41 square area in this direction may be used if certain directions have exceeded image border Only to get image border.ε can take 100.Find out ak,bkQ can be obtained according to formula (3)k, to all pixels point according to Output image Q can be obtained in above method solution, is (d) the corresponding output figure of video frame shown in (a) in Fig. 2 shown in (d) Picture indicates filtered and covers picture value.It should be noted that at this moment also needing to cover picture value divided by 255, its range is made to restore 0 to 1 Between.
It is exemplified below and input picture is filtered using guidance figure to obtain output image, and obtained according to output image Obtain the process for covering picture value of each pixel in target video frame.
Enabling radius is 1, i.e. wkThe square for multiplying 3 for 3, as shown in figure 3, each grid indicates that a picture point element, grid 0 are Pixel k to be filtered, grid 0-8 constitute 3 × 3 neighborhood wk, the ash of the digital representation pixel in grid bracket Y value in angle value, that is, YCbCr color space, (a) indicate guidance figure G in the neighborhood w of pixel kkInterior value (b) indicates Neighborhood w of the input picture P in pixel kkInterior value (c) indicates GP in the neighborhood w of pixel kkInterior value, calculation method Value for the pixel of G, P corresponding position is multiplied, and (d) is G2In the neighborhood w of pixel kkValue, calculation method is each in G Square of the value of pixel.
ThenFor neighborhood w in (c)kThe mean value of the value of interior all pixels point, calculated result 3985,For neighborhood w in (a)kThe mean value of the value of interior all pixels point, calculated result 55.4,For neighborhood in (b) wkThe mean value of the value of interior all pixels point, calculated result 71.2,For neighborhood w in (d)kThe value of interior all pixels point Mean value, calculated result 3104, then available ak,bk:
So far, available pixel k corresponding value Q in output image Qk:
Qk=akGk+bk=0.3 × 56+54.58 ≈ 71
As it can be seen that the picture value 70 of initially covering in input picture P becomes output by the filtering of guiding figure for pixel k Picture value 71 is covered in image Q.Each pixel can be obtained in output figure by calculating in the manner described above all pixels point As corresponding value in Q.QkPixel k after as filtering covers picture value, by QkValue divided by 255 make its normalize to [0,1] it Between to get to pixel k cover picture value.
In practical application, inevitably occurs noise and impurity, that is, noise in target video frame, make an uproar to eliminate The interference of point and impurity, can also be adjusted the picture value of covering being calculated.Specifically, can be according to following calculation formula tune Each pixel covers picture value in whole target video frame:
Wherein, α ' is that pixel covers picture value in target video frame adjusted, and α be this in target video frame before adjustment Pixel covers picture value.
Fig. 4 is to calculate functional image corresponding to the formula of α ', it can be seen that pass through such adjustment, it is lesser to cover picture value Become smaller, biggish picture value of covering becomes much larger, and benefit is: since that acquires at noise under normal circumstances covers picture value less than 0.5, That acquires at prospect covers picture value greater than 0.5, doing so can make the picture value of covering at noise become smaller, and prospect covers picture value It becomes much larger, to reduce influence of the noise to finally synthesizing, improves the accuracy of foreground extraction.
S104 covers picture value according to each pixel, determines each pixel in the foreground image of target video frame Third rgb value obtains the foreground image of target video frame.
Obtain each pixel cover picture value after, picture value, the first rgb value, the second rgb value are covered according to pixel, can To obtain the third rgb value of pixel, thus the foreground image to target video frame.
It should be understood that the value range for covering picture value is [0,1], wherein indicated in target video frame when covering picture value and being 0 The percentage of foreground color is 0 in the color value of the pixel, i.e., the first rgb value of the pixel is equal to the second rgb value, namely The third rgb value F of the pixelB,FG,FRIt is 0；The color value of the pixel in target video frame is indicated when covering picture value and being 1 The percentage of middle foreground color is 100%, i.e. the first rgb value of the pixel is equal to third rgb value, FB=CB,FG=CG,FR= CR.And when covering picture value greater than 0 and when less than 1, indicate that pixel is likely to be at the edge of foreground image.
In fact, when cover picture value it is especially small when, if directly solving third rgb value using synthesizing equation, it will cause Biggish error can directly enable F at this timeB,FG,FRIt is 0, this is done because cover picture value very little, FB,FG,FRValue not The result of stingy figure can be impacted.
Therefore, in one implementation, above-mentioned that picture value is covered according to each pixel, determine each pixel described The step of third rgb value in the foreground image of target video frame, may include:
For each pixel, when the pixel is when covering picture value less than third predetermined threshold value, by the pixel in target Tri- component values of R, G, B of third rgb value in the foreground image of video frame images are disposed as zero, and third predetermined threshold value is small In 1 value；
Picture value is covered more than or equal to the third predetermined threshold value and when less than 1, or when the pixel when the pixel When covering picture value equal to 1, threeth RGB of the pixel in the foreground image of target video frame is calculated according to following calculation formula Value:
And
Wherein, FB,FG,FRRespectively indicate B, G, R of third rgb value of the pixel in the foreground image of target video frame Component value.
It should be understood that when pixel is when covering picture value equal to 1, according to above-mentioned calculation formula it is found that the of the pixel Three rgb values are equal to the first rgb value of the pixel.Since the value range of the rgb value of pixel is [0,255], conjunction is being used F is acquired at equationR,FG,FBAfterwards, it is also necessary to further by FR,FG,FBValue be limited between [0,255].In the present embodiment Three preset thresholds can be 0.04, naturally it is also possible to empirically carry out value, this reality to third predetermined threshold value with actual demand Example is applied not limit this.
Further, picture value is covered more than or equal to third predetermined threshold value and when less than 1 when pixel, the picture is being calculated After the third rgb value in the foreground image of target video frame, method provided in this embodiment can also include: vegetarian refreshments
In the case where the background image of target video frame is green curtain, the G component value in third rgb value is adjusted to third The average value of B component value and R component value in rgb value；
In the case where the background image of target video frame is blue curtain, the B component value in third rgb value is adjusted to third G component value in rgb value.
It should be understood that covering picture value more than or equal to third predetermined threshold value and less than 1, indicate that pixel is in foreground image Edge, and the pixel at the edge in foreground image is easy to happen color spillover when scratching figure.Therefore in order to solve before The color overflow problem of scape image border needs pair for covering picture value more than or equal to third predetermined threshold value and pixel less than 1 B component or G component in its third rgb value are adjusted.Specifically, enabling the F of the pixel when background image is green curtainG =(FB+FR)/2；When background image be Lan Mushi, enable the F of the pixelB=FG。
As it can be seen that be directed to color spillover existing in the prior art, the present embodiment cover picture value more than or equal to third it is pre- If threshold value and less than 1 in the case where, in the third rgb value F for acquiring pixelR,FG,FBIt later, and for background image is green curtain Or two kinds of situations of blue curtain, to third rgb value FR,FG,FBIt is adjusted, color spillover, raising pair can be effectively reduced in this way In the stingy figure effect of the small objects such as hairline, and calculation amount is smaller, can be handled in real time video, therefore, this reality The scheme for applying example offer can be applied in live scene.
In another implementation, above-mentioned that picture value is covered according to each pixel, determine each pixel in the mesh Mark video frame foreground image in third rgb value the step of, may include:
For each pixel, when the pixel is when covering picture value less than or equal to four preset thresholds, which is existed Tri- component values of R, G, B of third rgb value in the foreground image of target video frame are disposed as zero；
When the pixel is when covering picture value more than or equal to five preset thresholds, by the pixel target video frame prospect Third rgb value in image is set as the first rgb value of the pixel；Wherein, the 5th preset threshold is greater than the 4th preset threshold；
Picture value is covered greater than the 4th preset threshold and when less than five preset thresholds, according to third class pixel when the pixel First rgb value of point, determines third rgb value of the pixel in the foreground image of target video frame, wherein third class pixel Point is the pixel covered picture value in target video frame and be more than or equal to the 5th preset threshold.
It is understood that acquire each pixel cover picture value after, can directly using synthesis equation calculation it is each Third rgb value of the pixel in foreground image.But there are errors in the solution procedure due to covering picture value, directly utilize synthesis Equation solution third rgb value will cause certain mistake.It, can be using the method for neighborhood search in order to reduce color spillover Determine third rgb value.The thought of neighborhood search is the third rgb value for uncertain pixel, according in its preset range The third rgb value of the pixel that can determine estimate to obtain.
It is possible, firstly, to understand, it is less than or equal to the 4th preset threshold for covering picture value, and cover picture value more than or equal to the 5th The pixel of preset threshold, these pixels are not at the marginal portion of foreground image, and third rgb value influences stingy figure result Less.Therefore, the pixel for being less than or equal to the 4th preset threshold for covering picture value, can be directly by the 3rd RGB of the pixel Tri- component values of R, G, B of value are disposed as zero；The pixel for being more than or equal to the 5th preset threshold for covering picture value, can be direct Set the third rgb value of the pixel to the first rgb value of the pixel.
Wherein, the 4th preset threshold can take pre- close to 0 or the value equal to 0, such as 0,10/255,20/255, the 5th If threshold value can take close to 1 or the value equal to 1, such as 1,250/255,245/255, the 4th preset threshold and the 5th default threshold The value of value can rule of thumb be set with actual demand.
For covering picture value greater than the 4th preset threshold and less than the pixel of the 5th preset threshold, i.e. the 5th class pixel, Since it is in the marginal portion of foreground image, according to the first rgb value of third class pixel, can accurately estimate Count the third rgb value of these pixels.
In practical application, target pixel points can be determined from third class pixel, by the first rgb value of target pixel points It is determined as third rgb value of the pixel in the foreground image of target video frame.For example, can will be nearest apart from the pixel Third class pixel be determined as target pixel points.
It, can be using the pixel as starting point, according to the preset direction of search and step in a kind of preferable embodiment The long pixel traversed other than the pixel, the pixel that search first meets preset stopping search condition is determined as Target pixel points, wherein the preset condition that stops search are as follows: belong to third class pixel and the first corresponding rgb value So that DR,DG,DBThe sum of three's absolute value is less than the 6th preset threshold, wherein
DR=α C'R+(1-α)BR-C″R
DG=α C'G+(1-α)BG-C”G
DB=α C'B+(1-α)BB-C″B
C'B,C'G,C'RB, G, the R component value of the first rgb value of target pixel points are respectively indicated, α is covering for the pixel Picture value, C "BB,C”G,C″RRRespectively B, G of the first rgb value of the pixel, R component value.
For example, Fig. 5 shows two group searching directions, as shown in thick-line arrow in figure, x and y-axis are orthogonal, wherein (a) Showing first group of direction is to be total to four direction along x, positive direction of the y-axis and opposite direction, shows second group (b) as positive direction of the x-axis 45 °, 135 °, 225 ° and 315 ° four directions are successively rotated clockwise.
It for each 5th class pixel, is successively scanned for using two group searching directions, i.e., to first the 5th class picture When vegetarian refreshments uses first group, second group is used to second the 5th class pixel, first is used to the 5th class pixel of third Group uses second group to the 4th the 5th class of pixel, is so used alternatingly, two group searching directions are used alternatingly in this way to be increased Add the diversity of search, reduction is searched for always upwards in a prescription but searches for the mistake less than caused by.Every time when search, Ke Yiyi Secondary to scan in the direction of the clock in four direction, step-size in search can be set to 1, i.e. four direction increases a picture every time Vegetarian refreshments scans for.Referring to Fig. 6, (a) in Fig. 6, (b) are respectively that (a) in Fig. 5, the corresponding search of the direction of search shown in (b) are suitable Sequence, each square indicates a pixel in figure, marked as the 5th class picture that the square of A indicates third rgb value to be determined Vegetarian refreshments, digital representation search-order in figure are that is, every to increase a step-length needs and search for one time along four direction.
It should be noted that during being scanned for for current pixel point A, when certain direction reaches image border When, the search of the direction can be stopped, and continue to scan in the above described manner in other directions.When a certain picture searched Vegetarian refreshments A ' meet it is preset stop search condition when, then can stop the search for current pixel point A, at this point, by pixel A ' is used as target pixel points, and the third rgb value of pixel A is enabled to be equal to the first rgb value of pixel A ', i.e. FR=C'R,FG=C 'G,FB=G'B。
Further, determining that all pixels point after the third rgb value in the foreground image of target video frame, may be used also To be filtered according to the following formula to third rgb value of every one the 5th class pixel in the foreground image of the target video frame Wave processing:
Wherein, the 5th class pixel is that picture value is covered in target video frame greater than the 4th preset threshold and less than the 5th default threshold The pixel of value, FR', FG', FBThe 5th class pixel k' after ' respectively filtering processing is in the foreground image of target video frame Third rgb value, wk'It indicates centered on the 5th class pixel k', the neighborhood that is made of the pixel of preset quantity, αiTable Show neighborhood wk'In include ith pixel point cover picture value,Ith pixel point before being respectively filtered Third rgb value in the foreground image of target video frame.
It should be understood that third rgb value directly uses the first rgb value of target pixel points for the 5th class pixel, There may be mistakes for this mode.It is therefore possible to use the mode of weight filtering reduces the third rgb value of the 5th class pixel Mistake improves the stingy figure effect for small objects such as hairlines to effectively reduce color spillover.
For example, for the 5th class pixel k' to be filtered, if its neighborhood wk'Radius be 2, i.e. the 5th class pixel k' All pixels point up and down, in each 2 pixel point ranges in left and right belong to neighborhood wk', i.e. neighborhood wk'Include 25 pixels. The 5th class pixel k' is filtered according to above-mentioned formula using the third rgb value of this 25 pixels, it is available The more accurate third rgb value of pixel k'.
It, can also be to the background image of target video frame after having obtained the foreground image of target video frame in practical application It is replaced, i.e., foreground image and other background images is synthesized, obtain the replaced video frame of background.
Specifically, covering picture value according to each pixel in step S104, determine the pixel before target video frame After the step of third rgb value in scape image, this method can also include:
The second background image of the background image of preset replacement target video frame is obtained, and obtains the second background image 4th rgb value of each pixel；
According to each pixel of target video frame cover picture value, third rgb value, the second background image each pixel The 4th rgb value, determine the rgb value of each pixel of the replaced composograph of background, realize the background of target video frame Replacement.
Specifically, each pixel of target video frame can be covered picture value, third rgb value, the second background image 4th rgb value of each pixel substitutes into synthesis equation, and each pixel of the replaced composograph of background is calculated Rgb value.Wherein, the second background image can be the frame image in default video, or preset piece image, herein Without limitation.
In practical application, it may occur that the different situation of size of the second background image and target video frame, at this In the case of kind, the step of four rgb value of each pixel of above-mentioned the second background image of acquisition, may include:
Judge whether the size of the second background video is identical as the size of target video frame；
If so, obtaining the 4th rgb value of each pixel of the second background image；
Otherwise, the second background image is zoomed to, then second back scaled after identical as the size of target video frame 4th rgb value of each pixel of scape image.
It should be understood that carrying out background replacement if the second background image is different from the size of target video frame When mistake can occur, it is therefore desirable to the size for adjusting the second background image keeps it consistent with the size of target video frame.Specifically , it can use image scaling techniques and the second background image zoomed to, common scaling consistent with the size of target video frame Algorithm has bilinear interpolation and bicubic interpolation etc..
As seen from the above, in scheme provided in an embodiment of the present invention, foreground image preparation method provided in this embodiment and Device targetedly covers picture value according to corresponding to curtain type according to the curtain type of the background image of target video frame Calculation formula, obtain each pixel covers picture value, and background color can be effectively reduced to the influence of picture value is covered, to reduce Color spillover has reached preferably stingy figure effect.
Scheme provided in an embodiment of the present invention is illustrated with a specific embodiment below.Processing as shown in Figure 7 Flow chart, using original green curtain/indigo plant curtain video and the background video for replacing green curtain/indigo plant curtain background as input, final output is to close At video, it is possible to understand that, background replacement is carried out to all each frame images of original green curtain/indigo plant curtain video in order to realize, The frame number of background video should be more than or equal to original green curtain/indigo plant curtain video frame number, certainly, if the frame number of background video Less than original green curtain/indigo plant curtain video frame number, the original green curtain of multiframe/indigo plant curtain video frame can also be used into same background video frame It is replaced.Identical processing method is all made of to each frame of original green curtain/indigo plant curtain video in this programme.
The i-th frame original image in original green curtain/indigo plant curtain video is obtained first, to the background color of the i-th frame original image It extracts, determines second rgb value of each pixel in background image.Specifically, hsv color space Central Plains can be obtained Histogram of all pixels point of beginning image on tone H component, to estimate the tone H component value H of background imageB；Base In the tone H component value H of background imageB, it can be determined that background image is green curtain or blue curtain；When judging background image neither Green curtain is also not Lan Mushi, terminates the process flow of current video frame, carries out the processing of next frame original image.When judging to carry on the back Scape image is green curtain or Lan Mushi, can carry out background cutout for green curtain and blue curtain respectively, obtain foreground image.
When carrying out the processing of the i-th frame original image, the i-th frame background image in background video can be obtained simultaneously, and The i-th frame background image is handled, if such as the size of the i-th frame background image and the size of the i-th frame original image it is different It causes, then needing to scale the i-th frame background image keeps its size identical as green curtain/size of indigo plant curtain image, then by the i-th frame original graph The foreground image of picture carries out image with the i-th frame background image in background video and synthesizes.To each frame original image according to upper It states method and carries out background replacement, then according to the sequence of video frame in original video, export the replaced composograph institute of background The synthetic video of composition.
Below by the validity of the description of test embodiment of the present invention.As shown in figure 8, (a) is indicated in original green curtain video A frame image A, (b) indicate new background picture A ', the purpose of this experiment is that the green background in image A is substituted for A '. (c) (d) (e) indicates the result figure that background replacement is carried out using the method for the prior art, and (f) (g) (h) indicates real using the present invention The method for applying example offer carries out the result figure of background replacement: where (c) (f) indicates the result figure for covering picture value of pixel, here In order to facilitate display, picture value will be covered multiplied by 255, pure white corresponding 255, ater corresponds to 0；(d) (g) indicates the prospect obtained Image, since original image can be used as foreground image in the method for the prior art, so (d) being substantially exactly original figure As A；(e) (h) indicates the composite result after final replacement background image.
By the comparison of (c) (f) it is found that the picture value transition of covering of method provided in an embodiment of the present invention pixel obtained is put down It is sliding, and preferable processing result is also obtained at the details such as hairline.By the comparison of (d) (g) it is found that the prior art is direct Use original image as foreground image, and the information for the foreground image that method provided in an embodiment of the present invention obtains is compared with subject to Really, although in some regions, such as there are mistakes for the information of the corresponding foreground image of image upper right corner pixel, due to this Place covers that picture value is smaller, does not influence final composite result.By the comparison of (e) (h) it is found that the method for the prior art has significantly , that is, there is apparent color spillover in green background residual, and method provided in an embodiment of the present invention can effectively reduce face Color spillover, so that the foreground edge transition of composograph is very natural.
There is better processing result in order to clearly show scheme provided in an embodiment of the present invention compared with the existing technology, The original image of (a)~(h) in Fig. 8, as shown in figure 9, (a)~(h) in Fig. 9 is respectively and in Fig. 8 (a)~(h) it is corresponding.
Corresponding with above-mentioned foreground image preparation method, the embodiment of the invention also provides a kind of foreground images to be filled It sets.Corresponding with embodiment of the method shown in FIG. 1, Figure 10 is that a kind of foreground image provided in an embodiment of the present invention obtains device Structural schematic diagram, the apparatus may include:
Module 101 is obtained, for obtaining target video frame；Wherein, the target video frame is any in original video Frame image；
First determining module 102 determines every for the first rgb value according to each pixel of the target video frame The background image of second rgb value and the target video frame of one pixel in the background image of the target video frame Curtain type；
First obtains module 103, for the first rgb value and the second rgb value according to each pixel, according to the curtain Cloth type is corresponding to cover picture value calculation formula, and obtain each pixel covers picture value；
Second determining module 104 determines that each pixel is regarded in the target for covering picture value according to each pixel Third rgb value in the foreground image of frequency frame obtains the foreground image of the target video frame.
As seen from the above, in scheme provided in an embodiment of the present invention, to foreground image preparation method provided in this embodiment And device targetedly covers picture according to corresponding to curtain type according to the curtain type of the background image of target video frame It is worth calculation formula, obtain each pixel covers picture value, and background color can be effectively reduced to the influence of picture value is covered, to reduce Color spillover has reached preferably stingy figure effect.
Specifically, the curtain type can be green curtain or blue curtain；
Correspondingly, described first obtains module 103, can be used for:
In the case where the curtain type of the background image of the target video frame is green curtain, calculated according to following picture value of covering What formula calculated each pixel in the target video frame covers picture value:
In the case where the curtain type of the background image of the target video frame is blue curtain, calculated according to following picture value of covering What formula calculated each pixel in the target video frame covers picture value:
Wherein, what α indicated pixel in the target video frame covers picture value, CB,CG,CRRespectively indicate the of the pixel B, G of one rgb value, R component value, BB,BG,BRRespectively indicate of the pixel in the background image of the target video frame B, G of two rgb values, R component value.
Specifically, second determining module 104, can be used for:
For each pixel, when the pixel is when covering picture value less than third predetermined threshold value, by the pixel described Tri- component values of R, G, B of third rgb value in the foreground image of target video frame image are disposed as zero, third predetermined threshold value For the value less than 1；Picture value is covered more than or equal to the third predetermined threshold value and when less than 1 when the pixel, or works as the pixel When covering picture value equal to 1 of point, calculates the pixel in the foreground image of the target video frame according to following calculation formula Third rgb value:
And
Wherein, FB,FG,FRRespectively indicate third rgb value of the pixel in the foreground image of the target video frame B, G, R component value.
Specifically, described device can also include:
The first adjustment module, for counting to pixel of the picture value more than or equal to the third predetermined threshold value and less than 1 is covered Calculation obtains the pixel after the third rgb value in the foreground image of the target video frame, in the target video frame In the case that background image is green curtain, the G component value in the third rgb value is adjusted to the B component in the third rgb value The average value of value and R component value；In the case where the background image of the target video frame is blue curtain, by the third rgb value In B component value be adjusted to the G component value in the third rgb value.
Specifically, first determining module 102, may include:
First obtains submodule, for the first rgb value according to each pixel of the target video frame, obtains each The tone H component value of pixel；Wherein, the first rgb value institute of the pixel is true according to the tone H component value of any pixel point Fixed value；
It determines submodule, for the tone H component value according to each pixel, determines that each pixel is regarded in the target The second rgb value in the background image of frequency frame.
Specifically, the determining submodule, may include:
Statistic unit, for counting the number of pixel corresponding to each tone H component value, most by the number of pixel Tone H component value of more tone H component values as the background image of the target video frame；
Judging unit judges the target view for the tone H component value according to the background image of the target video frame Whether the background image of frequency frame is green curtain or blue curtain；
First determination unit, for judging that the background image of the target video frame is green curtain in the judging unit In the case of, the average value of the first rgb value of the first kind pixel of the target video frame is determined as each pixel in institute State the second rgb value in the background image of target video frame, wherein the first kind pixel is tone H component value and green Pixel of the absolute value of the difference of corresponding tone value less than the first preset threshold；
Second determination unit, for judging that the background image of the target video frame is blue curtain in the judging unit In the case of, the average value of the first rgb value of the second class pixel of the target video frame is determined as each pixel in institute State the second rgb value in the background image of target video frame, wherein the second class pixel is tone H component value and blue Pixel of the absolute value of the difference of corresponding tone value less than the second preset threshold.
Specifically, described device can also include:
Second adjustment module determines each for covering picture value according to each pixel in second determining module 104 Pixel, according to following calculation formula, adjusts the mesh before the third rgb value in the foreground image of the target video frame Each pixel covers picture value in mark video frame:
Wherein, α ' is that pixel covers picture value in the target video frame adjusted, and α be that the target before adjustment regards The pixel covers picture value in frequency frame.
Specifically, described device can also include:
Second acquisition module determines the picture for covering picture value according to each pixel in second determining module 104 Vegetarian refreshments obtains the preset replacement target video frame after the third rgb value in the foreground image of the target video frame Background image the second background image, and obtain the 4th rgb value of each pixel of second background image；
Replacement module, for covering picture value, third rgb value, described according to each pixel of the target video frame 4th rgb value of each pixel of two background images determines the RGB of each pixel of the replaced composograph of background Value realizes the background replacement of the target video frame.
Specifically, described second obtains module, may include:
Second judgment submodule, for judge second background video size whether the ruler with the target video frame It is very little identical；If so, triggering second obtains submodule；Otherwise, triggering third obtains submodule；
Described second obtains submodule, the 4th rgb value of each pixel for obtaining second background image；
The third obtains submodule, for second background image to be zoomed to the size with the target video frame It is identical, then second background image after being scaled each pixel the 4th rgb value.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device reality For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method Part explanation.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention It is interior.

Claims (18)

1. a kind of foreground image preparation method, which is characterized in that the described method includes:
Obtain target video frame；Wherein, the target video frame is any frame image in original video；
According to the first rgb value of each pixel of the target video frame, determine each pixel in the target video frame Background image in the second rgb value and the target video frame background image curtain type；
According to the first rgb value and the second rgb value of each pixel, public affairs are calculated according to picture value of covering corresponding with the curtain type Formula, obtain each pixel covers picture value；
Picture value is covered according to each pixel, determines third of each pixel in the foreground image of the target video frame Rgb value obtains the foreground image of the target video frame.
2. the method according to claim 1, wherein the curtain type is green curtain or blue curtain；
First rgb value and the second rgb value according to each pixel covers picture value meter according to corresponding with the curtain type Formula is calculated, the step of covering picture value of each pixel is obtained, comprising:
In the case where the curtain type of the background image of the target video frame is green curtain, picture value calculation formula is covered according to following Calculate each pixel in the target video frame covers picture value:
In the case where the curtain type of the background image of the target video frame is blue curtain, picture value calculation formula is covered according to following Calculate each pixel in the target video frame covers picture value:
Wherein, what α indicated pixel in the target video frame covers picture value, CB,CG,CRRespectively indicate the first RGB of the pixel B, G of value, R component value, BB,BG,BRRespectively indicate twoth RGB of the pixel in the background image of the target video frame B, G of value, R component value.
3. method according to claim 1 or 2, which is characterized in that the picture value of covering according to each pixel determines every The step of third rgb value of one pixel in the foreground image of the target video frame, comprising:
For each pixel, when the pixel is when covering picture value less than third predetermined threshold value, by the pixel in the target Tri- component values of R, G, B of third rgb value in the foreground image of video frame images are disposed as zero, and third predetermined threshold value is small In 1 value；
When the picture value of covering of the pixel covers picture more than or equal to the third predetermined threshold value and when less than 1, or when the pixel When value is equal to 1, threeth RGB of the pixel in the foreground image of the target video frame is calculated according to following calculation formula Value:
And
Wherein, FB,FG,FRRespectively indicate B, G, R of third rgb value of the pixel in the foreground image of the target video frame Component value.
4. according to the method described in claim 3, it is characterized in that, described when the picture value of covering of the pixel is more than or equal to described the Three preset thresholds and when less than 1, are being calculated third rgb value of the pixel in the foreground image of the target video frame Later, the method also includes:
In the case where the background image of the target video frame is green curtain, the G component value in the third rgb value is adjusted to The average value of B component value and R component value in the third rgb value；
In the case where the background image of the target video frame is blue curtain, the B component value in the third rgb value is adjusted to G component value in the third rgb value.
5. the method according to claim 1, wherein being determined every in the picture value of covering according to each pixel Before one pixel is the third rgb value in the foreground image of the target video frame the step of, the method also includes:
According to following calculation formula, adjust each pixel in the target video frame covers picture value:
Wherein, α ' is that pixel covers picture value in the target video frame adjusted, and α be the target video frame before adjusting In the pixel cover picture value.
6. the method according to claim 1, wherein each pixel according to the target video frame First rgb value, the step of determining the second rgb value of each pixel in the background image of the target video frame, comprising:
According to the first rgb value of each pixel of the target video frame, the tone H component value of each pixel is obtained；Its In, value determined by the first rgb value of the pixel according to the tone H component value of any pixel point；
According to the tone H component value of each pixel, determine each pixel in the background image of the target video frame Second rgb value.
7. according to the method described in claim 6, it is characterized in that, the tone H component value according to each pixel, determines The step of the second rgb value of each pixel in the background image of the target video frame, comprising:
The number for counting pixel corresponding to each tone H component value makees the largest number of tone H component values of pixel For the tone H component value of the background image of the target video frame；
According to the tone H component value of the background image of the target video frame, judge that the background image of the target video frame is No is green curtain or blue curtain；
In the case where the background image of the target video frame is green curtain, by the first kind pixel of the target video frame The average value of first rgb value is determined as second rgb value of each pixel in the background image of the target video frame, In, the first kind pixel is the absolute value of the difference of tone H component value tone value corresponding with green less than the first default threshold The pixel of value；
In the case where the background image of the target video frame is blue curtain, by the second class pixel of the target video frame The average value of first rgb value is determined as second rgb value of each pixel in the background image of the target video frame, In, the second class pixel is the absolute value of the difference of tone H component value tone value corresponding with blue less than the second default threshold The pixel of value.
8. the method according to claim 1, wherein determining should in the picture value of covering according to each pixel After pixel is the third rgb value in the foreground image of the target video frame the step of, the method also includes:
The second background image of the background image of the preset replacement target video frame is obtained, and obtains second Background 4th rgb value of each pixel of picture；
According to each pixel of the target video frame cover picture value, third rgb value, second background image each picture 4th rgb value of vegetarian refreshments determines the rgb value of each pixel of the replaced composograph of background, realizes the target video The background of frame is replaced.
9. according to the method described in claim 8, it is characterized in that, each pixel for obtaining second background image Four rgb values the step of, comprising:
Judge whether the size of second background video is identical as the size of the target video frame；
If so, obtaining the 4th rgb value of each pixel of second background image；
Otherwise, second background image is zoomed to, then institute scaled after identical as the size of the target video frame State the 4th rgb value of each pixel of the second background image.
10. a kind of foreground image obtains device, which is characterized in that described device includes:
Module is obtained, for obtaining target video frame；Wherein, the target video frame is any frame image in original video；
First determining module determines each pixel for the first rgb value according to each pixel of the target video frame The curtain class of the background image of the second rgb value and the target video frame in the background image of the target video frame Type；
First obtains module, for the first rgb value and the second rgb value according to each pixel, according to the curtain type Corresponding to cover picture value calculation formula, obtain each pixel covers picture value；
Second determining module determines each pixel in the target video frame for covering picture value according to each pixel Third rgb value in foreground image obtains the foreground image of the target video frame.
11. device according to claim 10, which is characterized in that the curtain type is green curtain or blue curtain；
Described first obtains module, is used for:
In the case where the curtain type of the background image of the target video frame is green curtain, picture value calculation formula is covered according to following Calculate each pixel in the target video frame covers picture value:
In the case where the curtain type of the background image of the target video frame is blue curtain, picture value calculation formula is covered according to following Calculate each pixel in the target video frame covers picture value:
Wherein, what α indicated pixel in the target video frame covers picture value, CB,CG,CRRespectively indicate the first RGB of the pixel B, G of value, R component value, BB,BG,BRRespectively indicate twoth RGB of the pixel in the background image of the target video frame B, G of value, R component value.
12. device described in 0 or 11 according to claim 1, which is characterized in that second determining module is used for:
For each pixel, when the pixel is when covering picture value less than third predetermined threshold value, by the pixel in the target Tri- component values of R, G, B of third rgb value in the foreground image of video frame images are disposed as zero, and third predetermined threshold value is small In 1 value；Picture value is covered more than or equal to the third predetermined threshold value and when less than 1, or when the pixel when the pixel When covering picture value equal to 1, third of the pixel in the foreground image of the target video frame is calculated according to following calculation formula Rgb value:
And
Wherein, FB,FG,FRRespectively indicate B, G, R of third rgb value of the pixel in the foreground image of the target video frame Component value.
13. device according to claim 12, which is characterized in that described device further include:
The first adjustment module, for calculating to pixel of the picture value more than or equal to the third predetermined threshold value and less than 1 is covered To the pixel after the third rgb value in the foreground image of the target video frame, in the background of the target video frame In the case that image is green curtain, by the G component value in the third rgb value be adjusted to B component value in the third rgb value and The average value of R component value；In the case where the background image of the target video frame is blue curtain, by the B in the third rgb value Component value is adjusted to the G component value in the third rgb value.
14. device according to claim 10, which is characterized in that first determining module, comprising:
First obtains submodule, for the first rgb value according to each pixel of the target video frame, obtains each pixel The tone H component value of point；Wherein, according to the tone H component value of any pixel point determined by the first rgb value of the pixel Value；
It determines submodule, for the tone H component value according to each pixel, determines each pixel in the target video frame Background image in the second rgb value.
15. device according to claim 14, which is characterized in that the determining submodule, comprising:
Statistic unit, for counting the number of pixel corresponding to each tone H component value, by the largest number of of pixel Tone H component value of the tone H component value as the background image of the target video frame；
Judging unit judges the target video frame for the tone H component value according to the background image of the target video frame Background image whether be green curtain or blue curtain；
First determination unit, for judging the case where background image of the target video frame is green curtain in the judging unit Under, the average value of the first rgb value of the first kind pixel of the target video frame is determined as each pixel in the mesh Mark the second rgb value in the background image of video frame, wherein the first kind pixel is that tone H component value is corresponding with green Tone value absolute value of the difference less than the first preset threshold pixel；
Second determination unit, for judging the case where background image of the target video frame is blue curtain in the judging unit Under, the average value of the first rgb value of the second class pixel of the target video frame is determined as each pixel in the mesh Mark the second rgb value in the background image of video frame, wherein the second class pixel is that tone H component value is corresponding with blue Tone value absolute value of the difference less than the second preset threshold pixel.
16. device according to claim 10, which is characterized in that described device further include:
Second adjustment module determines each pixel for covering picture value according to each pixel in second determining module Before the third rgb value in the foreground image of the target video frame, according to following calculation formula, the target video is adjusted Each pixel covers picture value in frame:
Wherein, α ' is that pixel covers picture value in the target video frame adjusted, and α be the target video frame before adjusting In the pixel cover picture value.
17. device according to claim 10, which is characterized in that described device further include:
Second acquisition module determines that the pixel exists for covering picture value according to each pixel in second determining module After third rgb value in the foreground image of the target video frame, the background of the preset replacement target video frame is obtained Second background image of image, and obtain the 4th rgb value of each pixel of second background image；
Replacement module, for covering picture value, third rgb value, second back according to each pixel of the target video frame 4th rgb value of each pixel of scape image determines the rgb value of each pixel of the replaced composograph of background, real The background replacement of the existing target video frame.
18. device according to claim 17, which is characterized in that described second obtains module, comprising:
Second judgment submodule, for judge second background video size whether the size phase with the target video frame Together；If so, triggering second obtains submodule；Otherwise, triggering third obtains submodule；
Described second obtains submodule, the 4th rgb value of each pixel for obtaining second background image；
The third obtains submodule, for second background image to be zoomed to the size phase with the target video frame Together, the 4th rgb value of each pixel of second background image then after being scaled.
CN201710351704.7A 2017-05-18 2017-05-18 Foreground image obtaining method and device Active CN108961299B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710351704.7A CN108961299B (en) 2017-05-18 2017-05-18 Foreground image obtaining method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710351704.7A CN108961299B (en) 2017-05-18 2017-05-18 Foreground image obtaining method and device

Publications (2)

Publication Number Publication Date
CN108961299A true CN108961299A (en) 2018-12-07
CN108961299B CN108961299B (en) 2021-03-02

Family

ID=64461483

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710351704.7A Active CN108961299B (en) 2017-05-18 2017-05-18 Foreground image obtaining method and device

Country Status (1)

Country Link
CN (1) CN108961299B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919836A (en) * 2019-03-20 2019-06-21 广州华多网络科技有限公司 Video keying processing method, video keying processing client and readable storage medium storing program for executing
CN110111347A (en) * 2019-04-19 2019-08-09 深圳市华星光电技术有限公司 Logos extracting method, device and storage medium
CN110298783A (en) * 2019-06-05 2019-10-01 北京长立科技有限公司 One kind is scratched as method and system
CN111415367A (en) * 2020-03-18 2020-07-14 北京七维视觉传媒科技有限公司 Method and device for removing image background
CN113303905A (en) * 2021-05-26 2021-08-27 中南大学湘雅二医院 Interventional operation simulation method based on video image feedback
CN113781586A (en) * 2021-09-08 2021-12-10 广州光锥元信息科技有限公司 Method and system for adjusting skin color of human skin area in image

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1033681A2 (en) * 1999-03-04 2000-09-06 Xerox Corporation Method and apparatus for image data compression
CN102193651A (en) * 2010-03-11 2011-09-21 株式会社理光 Equipment, method and system for identifying laser point
CN102693537A (en) * 2011-01-17 2012-09-26 三星泰科威株式会社 Image surveillance system and method of detecting whether object is left behind or taken away
CN103955918A (en) * 2014-04-03 2014-07-30 吉林大学 Full-automatic fine image matting device and method
CN104200470A (en) * 2014-08-29 2014-12-10 电子科技大学 Blue screen image-matting method
US9042757B2 (en) * 2013-03-05 2015-05-26 Canon Kabushiki Kaisha Image forming apparatus
CN105243670A (en) * 2015-10-23 2016-01-13 北京航空航天大学 Sparse and low-rank joint expression video foreground object accurate extraction method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1033681A2 (en) * 1999-03-04 2000-09-06 Xerox Corporation Method and apparatus for image data compression
CN102193651A (en) * 2010-03-11 2011-09-21 株式会社理光 Equipment, method and system for identifying laser point
CN102693537A (en) * 2011-01-17 2012-09-26 三星泰科威株式会社 Image surveillance system and method of detecting whether object is left behind or taken away
US9042757B2 (en) * 2013-03-05 2015-05-26 Canon Kabushiki Kaisha Image forming apparatus
CN103955918A (en) * 2014-04-03 2014-07-30 吉林大学 Full-automatic fine image matting device and method
CN104200470A (en) * 2014-08-29 2014-12-10 电子科技大学 Blue screen image-matting method
CN105243670A (en) * 2015-10-23 2016-01-13 北京航空航天大学 Sparse and low-rank joint expression video foreground object accurate extraction method

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109919836A (en) * 2019-03-20 2019-06-21 广州华多网络科技有限公司 Video keying processing method, video keying processing client and readable storage medium storing program for executing
CN110111347A (en) * 2019-04-19 2019-08-09 深圳市华星光电技术有限公司 Logos extracting method, device and storage medium
CN110111347B (en) * 2019-04-19 2021-04-27 Tcl华星光电技术有限公司 Image sign extraction method, device and storage medium
CN110298783A (en) * 2019-06-05 2019-10-01 北京长立科技有限公司 One kind is scratched as method and system
CN110298783B (en) * 2019-06-05 2024-04-19 苏州常立科技有限公司 Image matting method and system
CN111415367A (en) * 2020-03-18 2020-07-14 北京七维视觉传媒科技有限公司 Method and device for removing image background
CN113303905A (en) * 2021-05-26 2021-08-27 中南大学湘雅二医院 Interventional operation simulation method based on video image feedback
CN113781586A (en) * 2021-09-08 2021-12-10 广州光锥元信息科技有限公司 Method and system for adjusting skin color of human skin area in image
CN113781586B (en) * 2021-09-08 2023-12-08 广州光锥元信息科技有限公司 Method and system for adjusting skin color of human skin area in image

Also Published As

Publication number Publication date
CN108961299B (en) 2021-03-02

Similar Documents

Publication Publication Date Title
CN108965647A (en) A kind of foreground image preparation method and device
CN108961299A (en) A kind of foreground image preparation method and device
Huang et al. Efficient contrast enhancement using adaptive gamma correction with weighting distribution
JP4335565B2 (en) Method and apparatus for detecting and / or tracking one or more color regions in an image or sequence of images
CN105187810B (en) A kind of auto white balance method and electronic medium device based on face color character
CN107180415B (en) Skin beautifying processing method and device in image
Ciurea et al. A large image database for color constancy research
DE102019106252A1 (en) Method and system for light source estimation for image processing
US8111946B2 (en) Image processing apparatus, imaging apparatus, and image processing method
CN110009588B (en) Portrait image color enhancement method and device
CN108537155A (en) Image processing method, device, electronic equipment and computer readable storage medium
CN107038680A (en) The U.S. face method and system that adaptive optical shines
CN106897981A (en) A kind of enhancement method of low-illumination image based on guiding filtering
CN103606132A (en) Multiframe digital image denoising method based on space domain and time domain combination filtering
JP2010508571A (en) Digital image processing using face detection and skin tone information
CN107770521B (en) Camera shooting mode method of adjustment
CN106780417A (en) A kind of Enhancement Method and system of uneven illumination image
Sidike et al. Adaptive trigonometric transformation function with image contrast and color enhancement: Application to unmanned aerial system imagery
CN113132696B (en) Image tone mapping method, image tone mapping device, electronic equipment and storage medium
CN109754375A (en) Image processing method, system, computer equipment, storage medium and terminal
CN110852956A (en) Method for enhancing high dynamic range image
CN114862698B (en) Channel-guided real overexposure image correction method and device
CN109829925A (en) A kind of method and model training method for extracting clean prospect in scratching figure task
CN108961258A (en) A kind of foreground image preparation method and device
CN106550227A (en) A kind of image saturation method of adjustment and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant