CN108230243A - One kind is based on salient region detection model background-blurring method - Google Patents
One kind is based on salient region detection model background-blurring method Download PDFInfo
- Publication number
- CN108230243A CN108230243A CN201810133575.9A CN201810133575A CN108230243A CN 108230243 A CN108230243 A CN 108230243A CN 201810133575 A CN201810133575 A CN 201810133575A CN 108230243 A CN108230243 A CN 108230243A
- Authority
- CN
- China
- Prior art keywords
- layer
- size
- convolutional
- output
- layers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 title claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims abstract description 21
- 238000005457 optimization Methods 0.000 claims abstract description 4
- 230000004913 activation Effects 0.000 claims description 72
- 238000010606 normalization Methods 0.000 claims description 12
- 239000000284 extract Substances 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 235000013399 edible fruits Nutrition 0.000 claims description 3
- 239000000463 material Substances 0.000 abstract description 3
- 230000000052 comparative effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Toys (AREA)
Abstract
The invention discloses one kind based on salient region detection model background-blurring method, include the following steps：Obtain original image, structure salient region detection model convolutional network obtains the Saliency maps of original image, the Saliency maps picture of acquisition is put into full condition of contact random field and is trained the Saliency maps picture after being optimized, the Saliency maps after optimization are subjected to binaryzation again or dividing processing obtains 01 matrix, obtain prospect index matrix and background index matrix；The overall situation for realizing original image using distance weighted average algorithm obscures；Finally prospect artwork is mutually spliced with blurred background figure, generation virtualization Background.The present invention can accurately not only detect complete salient region, and conspicuousness boundary is more clear, so as to retain the feature of foreground picture when background is blurred, not damage foreground picture picture material.
Description
Technical field
The present invention relates to digital image processing techniques fields, more particularly to a kind of to be based on salient region detection model background
Weakening method.
Background technology
Image background virtualization is processing procedure very common in the tasks such as image rendering, beautification, enhancing, it can very effectively
Ground protrudes target object and fading background information, so as to promote visual effect.At present, certain image processing softwares are preferably completed
This processing, but its processing method is required for carrying out foreground area manual mark, needs to expend a large amount of manpower, be not easy to
Mass disposal；In addition, the diffusion way that the prior art obscures is regular shape, it is difficult to be adapted in image complicated and changeable
Hold.Existing automatic background virtualization technology is immature in foreground edge extraction, causes obscure boundary clear, cuts to the region of mistake
Etc..
Invention content
For obscure boundary existing for existing background-blurring method it is clear the problems such as, the present invention proposes a kind of to be based on conspicuousness area
Domain detection model background-blurring method, can detect entire salient region, comprising multiple saliency objects, include small scale
The Various Complexes such as saliency object situation shows well, can accurately not only detect complete salient region, and significantly
Property boundary is more clear.So as to which the feature of foreground picture can be retained when background is blurred, foreground picture picture material is not damaged.
To achieve the above object, the technical scheme is that：One kind is background blurring based on salient region detection model
Method includes the following steps:
Step S1：Obtain original image；
Step S2：Salient region detection model is built based on convolutional neural networks, obtains the Saliency maps of original image；
Step S3：Saliency maps are put into full condition of contact random field to be trained, the Saliency maps after being optimized;
Step S4：Saliency maps after optimization are carried out with binaryzation or dividing processing obtains 01 matrix SBM, obtains prospect index square
Battle array IF and background index matrix IB, is defined as follows：
Wherein, M × N is all 1's matrix identical with original image resolution；
Step S5：The overall situation for realizing original image using distance weighted average algorithm obscures, and obtains original fuzzy graph；
Step S6：Utilization prospects index matrix IF extracts clear foreground picture to original image, using background index matrix IB to original
Beginning fuzzy graph extracts blurred background figure；Finally clear foreground picture with blurred background figure is mutually spliced, obtains background blurring result.
Further, the specific network structure of salient region detection model is as follows：
First layer is input layer, inputs original image；
The second layer is made of two convolutional layers, wherein first convolutional layer, using 64 convolution kernels, size is（4,4,3）, second
A convolutional layer uses 64 convolution kernels, and size is（3,3,64）, activation primitive is ReLU functions；
Third layer is pond layer, and size is（2,2）, activation primitive is ReLU functions；
4th layer is made of two convolutional layers, wherein first convolutional layer, using 128 convolution kernels, size is（3,3,64）, the
Two convolutional layers use 128 convolution kernels, and size is（3,3,128）, activation primitive is ReLU functions；
Layer 5 is pond layer, and size is（2,2）, activation primitive is ReLU functions；
Layer 6 is made of three convolutional layers, wherein first convolutional layer, using 256 convolution kernels, size is（3,3,128）, the
Two convolutional layers use 256 convolution kernels, and size is（3,3,256）, third convolutional layer uses 256 convolution kernels, and size is
（3,3,256）, activation primitive is ReLU functions；
Layer 7 is pond layer, and size is（2,2）, activation primitive is ReLU functions；
8th layer is made of three convolutional layers, wherein first convolutional layer, using 512 convolution kernels, size is（3,3,256）, the
Two convolutional layers use 512 convolution kernels, and size is（3,3,512）, third convolutional layer uses 512 convolution kernels, and size is
（3,3,512）, activation primitive is ReLU functions；
9th layer is pond layer, and size is（2,2）, activation primitive is ReLU functions；
Tenth layer is made of three convolutional layers, wherein first convolutional layer, using 512 convolution kernels, size is（3,3,512）, the
Two convolutional layers use 512 convolution kernels, and size is（3,3,512）, third convolutional layer uses 512 convolution kernels, and size is
（3,3,512）, activation primitive is ReLU functions；
Eleventh floor is pond layer, and size is（3,3）, it is 1 to expand marginal dimension, and activation primitive is ReLU functions；
Floor 12 is made of two convolutional layers, wherein first convolutional layer, using 1024 convolution kernels, size is（3,3,
512）, second convolutional layer be using 512 convolution kernels, size（3,3,1024）, activation primitive is ReLU functions；
13rd layer is made of two convolutional layers and a normalization layer, wherein first convolutional layer uses 256 convolution kernels, ruler
It is very little to be（3,3,1024）, second convolutional layer be using 512 convolution kernels, size（3,3,256）, activation primitive is ReLU letters
Number；
14th layer is module of deconvoluting, and two of which input is respectively that the 13rd layer of output is exported with Floor 12；
15th layer is module of deconvoluting, and two of which input is respectively the 14th layer of output and the 8th layer of output；
16th layer is module of deconvoluting, and two of which input is respectively that the 15th layer of output is exported with layer 6；
17th layer is module of deconvoluting, and two of which input is respectively the 16th layer of output and the 4th layer of output；
18th layer is module of deconvoluting, and two of which input is respectively that the 17th layer of output is exported with the second layer；
19th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 14th layer
Output, using 1 convolution kernel, size is（4,4,512）, the input of convolutional layer is the 14th layer of output, using 1 convolution kernel,
Size is（1,1,512）, for cascading layers by the output of deconvolute layer and convolutional layer into row of channels connection, activation primitive is ReLU letters
Number；
20th layer is made of two layers that deconvolute with a shear layer, wherein first layer that deconvolutes uses 1 convolution kernel, ruler
It is very little to be（8,8,2）, using 1 convolution kernel, size is second layer that deconvolutes（8,8,1）, will be deconvoluted layer knot using shear layer
Fruit cuts into the size as original image, and activation primitive is Sigmoid functions；
Second eleventh floor is made of the layer that deconvolutes, a convolutional layer, a cascading layers, the input for the layer that deconvolutes is the tenth
Nine layers of output, using 2 convolution kernels, size is（4,4,2）, the input of convolutional layer is the 15th layer of output, uses 1 convolution
Core, size are（1,1,512）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
Second Floor 12 is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（16,16,1）, layer result of deconvoluting is cut into size as original image using shear layer, activation primitive is
Sigmoid functions；
23rd layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
One layer of output, using 3 convolution kernels, size is（4,4,3）, the input of convolutional layer is the 16th layer of output, uses 1 convolution
Core, size are（1,1,256）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
24th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（8,8,2）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
25th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
Three layers of output, using 4 convolution kernels, size is（4,4,4）, the input of convolutional layer is the 17th layer of output, uses 1 convolution
Core, size are（1,1,128）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
26th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（4,4,3）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
27th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
Five layers of output, using 3 convolution kernels, size is（4,4,3）, the input of convolutional layer is the 17th layer of output, uses 1 convolution
Core, size are（1,1,256）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
28th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（2,2,4）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
29th layer is made of a cascading layers and a convolutional layer, cascading layers by the 28th layer, the 26th layer, second
14 layers, the second Floor 12, the 20th layer export and connected into row of channels, convolutional layer uses 1 convolution kernel, and size is（1,1,5）,
Activation primitive is Sigmoid functions, obtains final output result；
The module of deconvoluting is made of the layer that deconvolutes, a shear layer, Eltwise layers with normalization layer, specific to tie
Structure is as follows：If input is respectively characteristic pattern C1With characteristic pattern C2, size is respectively（h1, w1, k1）With（h2, w2, k2）And feature
Scheme C1Size be less than characteristic pattern C2Size, first layer uses k to deconvolute layer2A convolution kernel, size are（4,4, k1）, swash
Function living is ReLU functions, and input is characterized figure C1；The second layer is shear layer, according to characteristic pattern C2Size it is defeated to last layer
Go out to be sheared, third layer is Eltwise layers, to characteristic pattern C2It is multiplied pixel-by-pixel with last layer output, activation primitive is
ReLU functions；4th layer is normalization layer, and operation is normalized to last layer output.
Further, the step S3 is specifically included：
Output after condition of contact random field obtains Saliency maps convolution by way of connecting entirely entirely, by this output result
Input condition random field, if x=(x1, x2 ..., xn) is allowed to represent observed input data sequence, y=(y1, y2 ..., yn) table
Show a status switch, in the case where giving a list entries, the joint item of the CRF model definition status sequences of linear chain
Part probability is：
Wherein：Z is the probability normalization factor using input data sequence x as condition；F is an arbitrary characteristic function；W is every
The weights of a characteristic function,It is stringent positive potential function.
Further, the step S5 is specifically included：
According to the difference of each pixel importance, give different weight numbers respectively and be averaged, to original image RGB tri-
The weighted average matrix of three picture element matrixs is obtained in a channel respectively, realizes that the overall situation of original image obscures.
Further, the step S6 is specifically included：
If original image and original fuzzy graph are expressed as IO and IB, clear foreground picture ICF and blurred background figure IBB are extracted：
Wherein, i is x-axis coordinate, and j is y-axis coordinate,
The superposition of clear foreground picture ICF and blurred background figure IBB is the background blurring result of final image.
Compared with prior art, the present invention has advantageous effect：
（1）The present invention can detect entire salient region, comprising multiple saliency objects, include small scale saliency object
Various Complexes situation is waited to show well；
（2）The present invention can accurately not only detect complete salient region, and conspicuousness boundary is more clear.So as in void
The feature of foreground picture can be retained when changing background, do not damage foreground picture picture material.
Description of the drawings
Fig. 1 is a kind of flow diagram based on salient region detection model background-blurring method of the present invention；
Fig. 2 is the Comparative result schematic diagram of one embodiment of the invention.
Specific embodiment
The present invention will be further described with reference to the accompanying drawings and embodiments.
As shown in Figure 1, the present invention provides one kind based on salient region detection model background-blurring method, including following step
Suddenly:
Step S1：Obtain original image；
Step S2：Salient region detection model is built based on convolutional neural networks, obtains the Saliency maps of original image；
Step S3：Saliency maps are put into full condition of contact random field to be trained, the Saliency maps after being optimized;
Step S4：Saliency maps after optimization are carried out with binaryzation or dividing processing obtains 01 matrix SBM, obtains prospect index square
Battle array IF and background index matrix IB, is defined as follows：
Wherein, M × N is all 1's matrix identical with original image resolution；
Step S5：The overall situation for realizing original image using distance weighted average algorithm obscures, and obtains original fuzzy graph；
Step S6：Utilization prospects index matrix IF extracts clear foreground picture to original image, using background index matrix IB to original
Beginning fuzzy graph extracts blurred background figure；Finally clear foreground picture with blurred background figure is mutually spliced, obtains background blurring result.
The specific network structure of salient region detection model is as follows：
First layer is input layer, inputs original image；
The second layer is made of two convolutional layers, wherein first convolutional layer, using 64 convolution kernels, size is（4,4,3）, second
A convolutional layer uses 64 convolution kernels, and size is（3,3,64）, activation primitive is ReLU functions；
Third layer is pond layer, and size is（2,2）, activation primitive is ReLU functions；
4th layer is made of two convolutional layers, wherein first convolutional layer, using 128 convolution kernels, size is（3,3,64）, the
Two convolutional layers use 128 convolution kernels, and size is（3,3,128）, activation primitive is ReLU functions；
Layer 5 is pond layer, and size is（2,2）, activation primitive is ReLU functions；
Layer 6 is made of three convolutional layers, wherein first convolutional layer, using 256 convolution kernels, size is（3,3,128）, the
Two convolutional layers use 256 convolution kernels, and size is（3,3,256）, third convolutional layer uses 256 convolution kernels, and size is
（3,3,256）, activation primitive is ReLU functions；
Layer 7 is pond layer, and size is（2,2）, activation primitive is ReLU functions；
8th layer is made of three convolutional layers, wherein first convolutional layer, using 512 convolution kernels, size is（3,3,256）, the
Two convolutional layers use 512 convolution kernels, and size is（3,3,512）, third convolutional layer uses 512 convolution kernels, and size is
（3,3,512）, activation primitive is ReLU functions；
9th layer is pond layer, and size is（2,2）, activation primitive is ReLU functions；
Tenth layer is made of three convolutional layers, wherein first convolutional layer, using 512 convolution kernels, size is（3,3,512）, the
Two convolutional layers use 512 convolution kernels, and size is（3,3,512）, third convolutional layer uses 512 convolution kernels, and size is
（3,3,512）, activation primitive is ReLU functions；
Eleventh floor is pond layer, and size is（3,3）, it is 1 to expand marginal dimension, and activation primitive is ReLU functions；
Floor 12 is made of two convolutional layers, wherein first convolutional layer, using 1024 convolution kernels, size is（3,3,
512）, second convolutional layer be using 512 convolution kernels, size（3,3,1024）, activation primitive is ReLU functions；
13rd layer is made of two convolutional layers and a normalization layer, wherein first convolutional layer uses 256 convolution kernels, ruler
It is very little to be（3,3,1024）, second convolutional layer be using 512 convolution kernels, size（3,3,256）, activation primitive is ReLU letters
Number；
14th layer is module of deconvoluting, and two of which input is respectively that the 13rd layer of output is exported with Floor 12；
15th layer is module of deconvoluting, and two of which input is respectively the 14th layer of output and the 8th layer of output；
16th layer is module of deconvoluting, and two of which input is respectively that the 15th layer of output is exported with layer 6；
17th layer is module of deconvoluting, and two of which input is respectively the 16th layer of output and the 4th layer of output；
18th layer is module of deconvoluting, and two of which input is respectively that the 17th layer of output is exported with the second layer；
19th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 14th layer
Output, using 1 convolution kernel, size is（4,4,512）, the input of convolutional layer is the 14th layer of output, using 1 convolution kernel,
Size is（1,1,512）, for cascading layers by the output of deconvolute layer and convolutional layer into row of channels connection, activation primitive is ReLU letters
Number；
20th layer is made of two layers that deconvolute with a shear layer, wherein first layer that deconvolutes uses 1 convolution kernel, ruler
It is very little to be（8,8,2）, using 1 convolution kernel, size is second layer that deconvolutes（8,8,1）, will be deconvoluted layer knot using shear layer
Fruit cuts into the size as original image, and activation primitive is Sigmoid functions；
Second eleventh floor is made of the layer that deconvolutes, a convolutional layer, a cascading layers, the input for the layer that deconvolutes is the tenth
Nine layers of output, using 2 convolution kernels, size is（4,4,2）, the input of convolutional layer is the 15th layer of output, uses 1 convolution
Core, size are（1,1,512）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
Second Floor 12 is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（16,16,1）, layer result of deconvoluting is cut into size as original image using shear layer, activation primitive is
Sigmoid functions；
23rd layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
One layer of output, using 3 convolution kernels, size is（4,4,3）, the input of convolutional layer is the 16th layer of output, uses 1 convolution
Core, size are（1,1,256）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
24th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（8,8,2）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
25th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
Three layers of output, using 4 convolution kernels, size is（4,4,4）, the input of convolutional layer is the 17th layer of output, uses 1 convolution
Core, size are（1,1,128）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
26th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（4,4,3）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
27th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
Five layers of output, using 3 convolution kernels, size is（4,4,3）, the input of convolutional layer is the 17th layer of output, uses 1 convolution
Core, size are（1,1,256）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
28th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（2,2,4）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
29th layer is made of a cascading layers and a convolutional layer, cascading layers by the 28th layer, the 26th layer, second
14 layers, the second Floor 12, the 20th layer export and connected into row of channels, convolutional layer uses 1 convolution kernel, and size is（1,1,5）,
Activation primitive is Sigmoid functions, obtains final output result；
The module of deconvoluting is made of the layer that deconvolutes, a shear layer, Eltwise layers with normalization layer, specific to tie
Structure is as follows：If input is respectively characteristic pattern C1With characteristic pattern C2, size is respectively（h1, w1, k1）With（h2, w2, k2）And feature
Scheme C1Size be less than characteristic pattern C2Size, first layer uses k to deconvolute layer2A convolution kernel, size are（4,4, k1）, swash
Function living is ReLU functions, and input is characterized figure C1；The second layer is shear layer, according to characteristic pattern C2Size to last layer export
It is sheared, third layer is Eltwise layers, to characteristic pattern C2It is multiplied pixel-by-pixel with last layer output, activation primitive is
ReLU functions；4th layer is normalization layer, and operation is normalized to last layer output.
The step S3 is specifically included：
Output after condition of contact random field obtains Saliency maps convolution by way of connecting entirely entirely, by this output result
Input condition random field, if x=(x1, x2 ..., xn) is allowed to represent observed input data sequence, y=(y1, y2 ..., yn) table
Show a status switch, in the case where giving a list entries, the joint item of the CRF model definition status sequences of linear chain
Part probability is：
Wherein：Z is the probability normalization factor using input data sequence x as condition；F is an arbitrary characteristic function；W is every
The weights of a characteristic function,It is stringent positive potential function.
The step S5 is specifically included：
According to the difference of each pixel importance, give different weight numbers respectively and be averaged, to original image RGB tri-
The weighted average matrix of three picture element matrixs is obtained in a channel respectively, realizes that the overall situation of original image obscures.
The step S6 is specifically included：
If original image and original fuzzy graph are expressed as IO and IB, clear foreground picture ICF and blurred background figure IBB are extracted：
Wherein, i is x-axis coordinate, and j is y-axis coordinate,
The superposition of clear foreground picture ICF and blurred background figure IBB is the background blurring result of final image.
Fig. 2 is the background blurring comparative result figure obtained using the method for the present invention, and the left side is artwork, and the right is background blurring
Result figure.
What has been described above is only a preferred embodiment of the present invention, and the present invention is not limited to embodiment of above.It is appreciated that
The oher improvements and changes that those skilled in the art directly export or associate under the premise of in the design for not departing from the present invention,
It is considered as being included within protection scope of the present invention.
Claims (5)
1. one kind is based on salient region detection model background-blurring method, which is characterized in that includes the following steps:
Step S1：Obtain original image；
Step S2：Salient region detection model is built based on convolutional neural networks, obtains the Saliency maps of original image；
Step S3：Saliency maps are put into full condition of contact random field to be trained, the Saliency maps after being optimized;
Step S4：Saliency maps after optimization are carried out with binaryzation or dividing processing obtains 01 matrix SBM, obtains prospect index square
Battle array IF and background index matrix IB, is defined as follows：
Wherein, M × N is all 1's matrix identical with original image resolution；
Step S5：The overall situation for realizing original image using distance weighted average algorithm obscures, and obtains original fuzzy graph；
Step S6：Utilization prospects index matrix IF extracts clear foreground picture to original image, using background index matrix IB to original
Beginning fuzzy graph extracts blurred background figure；Finally clear foreground picture with blurred background figure is mutually spliced, obtains background blurring result.
2. according to claim 1 be based on salient region detection model background-blurring method, which is characterized in that described aobvious
The work property specific network structure of region detection model is as follows：
First layer is input layer, inputs original image；
The second layer is made of two convolutional layers, wherein first convolutional layer, using 64 convolution kernels, size is（4,4,3）, second
A convolutional layer uses 64 convolution kernels, and size is（3,3,64）, activation primitive is ReLU functions；
Third layer is pond layer, and size is（2,2）, activation primitive is ReLU functions；
4th layer is made of two convolutional layers, wherein first convolutional layer, using 128 convolution kernels, size is（3,3,64）, the
Two convolutional layers use 128 convolution kernels, and size is（3,3,128）, activation primitive is ReLU functions；
Layer 5 is pond layer, and size is（2,2）, activation primitive is ReLU functions；
Layer 6 is made of three convolutional layers, wherein first convolutional layer, using 256 convolution kernels, size is（3,3,128）, the
Two convolutional layers use 256 convolution kernels, and size is（3,3,256）, third convolutional layer uses 256 convolution kernels, and size is
（3,3,256）, activation primitive is ReLU functions；
Layer 7 is pond layer, and size is（2,2）, activation primitive is ReLU functions；
8th layer is made of three convolutional layers, wherein first convolutional layer, using 512 convolution kernels, size is（3,3,256）, the
Two convolutional layers use 512 convolution kernels, and size is（3,3,512）, third convolutional layer uses 512 convolution kernels, and size is
（3,3,512）, activation primitive is ReLU functions；
9th layer is pond layer, and size is（2,2）, activation primitive is ReLU functions；
Tenth layer is made of three convolutional layers, wherein first convolutional layer, using 512 convolution kernels, size is（3,3,512）, the
Two convolutional layers use 512 convolution kernels, and size is（3,3,512）, third convolutional layer uses 512 convolution kernels, and size is
（3,3,512）, activation primitive is ReLU functions；
Eleventh floor is pond layer, and size is（3,3）, it is 1 to expand marginal dimension, and activation primitive is ReLU functions；
Floor 12 is made of two convolutional layers, wherein first convolutional layer, using 1024 convolution kernels, size is（3,3,
512）, second convolutional layer be using 512 convolution kernels, size（3,3,1024）, activation primitive is ReLU functions；
13rd layer is made of two convolutional layers and a normalization layer, wherein first convolutional layer uses 256 convolution kernels, ruler
It is very little to be（3,3,1024）, second convolutional layer be using 512 convolution kernels, size（3,3,256）, activation primitive is ReLU letters
Number；
14th layer is module of deconvoluting, and two of which input is respectively that the 13rd layer of output is exported with Floor 12；
15th layer is module of deconvoluting, and two of which input is respectively the 14th layer of output and the 8th layer of output；
16th layer is module of deconvoluting, and two of which input is respectively that the 15th layer of output is exported with layer 6；
17th layer is module of deconvoluting, and two of which input is respectively the 16th layer of output and the 4th layer of output；
18th layer is module of deconvoluting, and two of which input is respectively that the 17th layer of output is exported with the second layer；
19th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 14th layer
Output, using 1 convolution kernel, size is（4,4,512）, the input of convolutional layer is the 14th layer of output, using 1 convolution kernel,
Size is（1,1,512）, for cascading layers by the output of deconvolute layer and convolutional layer into row of channels connection, activation primitive is ReLU letters
Number；
20th layer is made of two layers that deconvolute with a shear layer, wherein first layer that deconvolutes uses 1 convolution kernel, ruler
It is very little to be（8,8,2）, using 1 convolution kernel, size is second layer that deconvolutes（8,8,1）, will be deconvoluted layer knot using shear layer
Fruit cuts into the size as original image, and activation primitive is Sigmoid functions；
Second eleventh floor is made of the layer that deconvolutes, a convolutional layer, a cascading layers, the input for the layer that deconvolutes is the tenth
Nine layers of output, using 2 convolution kernels, size is（4,4,2）, the input of convolutional layer is the 15th layer of output, uses 1 convolution
Core, size are（1,1,512）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
Second Floor 12 is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（16,16,1）, layer result of deconvoluting is cut into size as original image using shear layer, activation primitive is
Sigmoid functions；
23rd layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
One layer of output, using 3 convolution kernels, size is（4,4,3）, the input of convolutional layer is the 16th layer of output, uses 1 convolution
Core, size are（1,1,256）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
24th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（8,8,2）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
25th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
Three layers of output, using 4 convolution kernels, size is（4,4,4）, the input of convolutional layer is the 17th layer of output, uses 1 convolution
Core, size are（1,1,128）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
26th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（4,4,3）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
27th layer is made of the layer that deconvolutes, a convolutional layer, a cascading layers, and the input for the layer that deconvolutes is the 20th
Five layers of output, using 3 convolution kernels, size is（4,4,3）, the input of convolutional layer is the 17th layer of output, uses 1 convolution
Core, size are（1,1,256）, cascading layers by the output of deconvolute layer and convolutional layer into row of channels connect, activation primitive ReLU
Function；
28th layer is made of the layer that deconvolutes with a shear layer, wherein the layer that deconvolutes, using 1 convolution kernel, size is
（2,2,4）, layer result of deconvoluting is cut into size as original image, activation primitive Sigmoid using shear layer
Function；
29th layer is made of a cascading layers and a convolutional layer, cascading layers by the 28th layer, the 26th layer, second
14 layers, the second Floor 12, the 20th layer export and connected into row of channels, convolutional layer uses 1 convolution kernel, and size is（1,1,5）,
Activation primitive is Sigmoid functions, obtains final output result；
The module of deconvoluting is made of the layer that deconvolutes, a shear layer, Eltwise layers with normalization layer, specific to tie
Structure is as follows：If input is respectively characteristic pattern C1With characteristic pattern C2, size is respectively（h1, w1, k1）With（h2, w2, k2）And feature
Scheme C1Size be less than characteristic pattern C2Size, first layer uses k to deconvolute layer2A convolution kernel, size are（4,4, k1）, swash
Function living is ReLU functions, and input is characterized figure C1；The second layer is shear layer, according to characteristic pattern C2Size to last layer export
It is sheared, third layer is Eltwise layers, to characteristic pattern C2It is multiplied pixel-by-pixel with last layer output, activation primitive is
ReLU functions；4th layer is normalization layer, and operation is normalized to last layer output.
3. according to claim 1 be based on salient region detection model background-blurring method, which is characterized in that
The step S3 is specifically included：
Output after condition of contact random field obtains Saliency maps convolution by way of connecting entirely entirely, by this output result
Input condition random field, if x=(x1, x2 ..., xn) is allowed to represent observed input data sequence, y=(y1, y2 ..., yn) table
Show a status switch, in the case where giving a list entries, the joint item of the CRF model definition status sequences of linear chain
Part probability is：
Wherein：Z is the probability normalization factor using input data sequence x as condition；F is an arbitrary characteristic function；W is every
The weights of a characteristic function,It is stringent positive potential function.
4. according to claim 1 be based on salient region detection model background-blurring method, which is characterized in that the step
Rapid S5 is specifically included：
According to the difference of each pixel importance, give different weight numbers respectively and be averaged, to original image RGB tri-
The weighted average matrix of three picture element matrixs is obtained in a channel respectively, realizes that the overall situation of original image obscures.
5. according to claim 1 be based on salient region detection model background-blurring method, which is characterized in that the step
Rapid S6 is specifically included：
If original image and original fuzzy graph are expressed as IO and IB, clear foreground picture ICF and blurred background figure IBB are extracted：
Wherein,iIt isxAxial coordinate,jIt isyAxial coordinate,
The superposition of clear foreground picture ICF and blurred background figure IBB is the background blurring result of final image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810133575.9A CN108230243B (en) | 2018-02-09 | 2018-02-09 | Background blurring method based on salient region detection model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810133575.9A CN108230243B (en) | 2018-02-09 | 2018-02-09 | Background blurring method based on salient region detection model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108230243A true CN108230243A (en) | 2018-06-29 |
| CN108230243B CN108230243B (en) | 2021-04-27 |
Family
ID=62661349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810133575.9A Expired - Fee Related CN108230243B (en) | 2018-02-09 | 2018-02-09 | Background blurring method based on salient region detection model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108230243B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109089040A (en) * | 2018-08-20 | 2018-12-25 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus and terminal device |
| CN109618173A (en) * | 2018-12-17 | 2019-04-12 | 深圳Tcl新技术有限公司 | Video-frequency compression method, device and computer readable storage medium |
| CN109636764A (en) * | 2018-11-01 | 2019-04-16 | 上海大学 | A kind of image style transfer method based on deep learning and conspicuousness detection |
| CN109727264A (en) * | 2019-01-10 | 2019-05-07 | 南京旷云科技有限公司 | Image generating method, the training method of neural network, device and electronic equipment |
| CN111680702A (en) * | 2020-05-28 | 2020-09-18 | 杭州电子科技大学 | Method for realizing weak supervision image significance detection by using detection frame |
| CN111861867A (en) * | 2020-07-02 | 2020-10-30 | 泰康保险集团股份有限公司 | Image background blurring method and device |
| CN116582743A (en) * | 2023-07-10 | 2023-08-11 | 荣耀终端有限公司 | Shooting method, electronic equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105513105A (en) * | 2015-12-07 | 2016-04-20 | 天津大学 | Image background blurring method based on saliency map |
| CN107169954A (en) * | 2017-04-18 | 2017-09-15 | 华南理工大学 | A kind of image significance detection method based on parallel-convolution neutral net |
| CN107247952A (en) * | 2016-07-28 | 2017-10-13 | 哈尔滨工业大学 | The vision significance detection method for the cyclic convolution neutral net supervised based on deep layer |
| CN107564025A (en) * | 2017-08-09 | 2018-01-09 | 浙江大学 | A kind of power equipment infrared image semantic segmentation method based on deep neural network |
| CN107610141A (en) * | 2017-09-05 | 2018-01-19 | 华南理工大学 | A kind of remote sensing images semantic segmentation method based on deep learning |
-
2018
- 2018-02-09 CN CN201810133575.9A patent/CN108230243B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105513105A (en) * | 2015-12-07 | 2016-04-20 | 天津大学 | Image background blurring method based on saliency map |
| CN107247952A (en) * | 2016-07-28 | 2017-10-13 | 哈尔滨工业大学 | The vision significance detection method for the cyclic convolution neutral net supervised based on deep layer |
| CN107169954A (en) * | 2017-04-18 | 2017-09-15 | 华南理工大学 | A kind of image significance detection method based on parallel-convolution neutral net |
| CN107564025A (en) * | 2017-08-09 | 2018-01-09 | 浙江大学 | A kind of power equipment infrared image semantic segmentation method based on deep neural network |
| CN107610141A (en) * | 2017-09-05 | 2018-01-19 | 华南理工大学 | A kind of remote sensing images semantic segmentation method based on deep learning |
Non-Patent Citations (4)
| Title |
|---|
| EVAN SHELHAMER 等: "Fully Convolutional Networks for Semantic Segmentation", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| NIAN LIU 等: "DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
| ZHAOWEI CAI 等: "A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection", 《ARXIV:1607.07155V1》 * |
| 杜慧: "基于视觉显著性的空频域多特征的目标检测方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109089040A (en) * | 2018-08-20 | 2018-12-25 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus and terminal device |
| CN109089040B (en) * | 2018-08-20 | 2021-05-14 | Oppo广东移动通信有限公司 | Image processing method, image processing device and terminal equipment |
| CN109636764A (en) * | 2018-11-01 | 2019-04-16 | 上海大学 | A kind of image style transfer method based on deep learning and conspicuousness detection |
| CN109618173A (en) * | 2018-12-17 | 2019-04-12 | 深圳Tcl新技术有限公司 | Video-frequency compression method, device and computer readable storage medium |
| CN109618173B (en) * | 2018-12-17 | 2021-09-28 | 深圳Tcl新技术有限公司 | Video compression method, device and computer readable storage medium |
| CN109727264A (en) * | 2019-01-10 | 2019-05-07 | 南京旷云科技有限公司 | Image generating method, the training method of neural network, device and electronic equipment |
| CN111680702A (en) * | 2020-05-28 | 2020-09-18 | 杭州电子科技大学 | Method for realizing weak supervision image significance detection by using detection frame |
| CN111680702B (en) * | 2020-05-28 | 2022-04-01 | 杭州电子科技大学 | Method for realizing weak supervision image significance detection by using detection frame |
| CN111861867A (en) * | 2020-07-02 | 2020-10-30 | 泰康保险集团股份有限公司 | Image background blurring method and device |
| CN111861867B (en) * | 2020-07-02 | 2024-02-13 | 泰康保险集团股份有限公司 | Image background blurring method and device |
| CN116582743A (en) * | 2023-07-10 | 2023-08-11 | 荣耀终端有限公司 | Shooting method, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108230243B (en) | 2021-04-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108230243A (en) | One kind is based on salient region detection model background-blurring method | |
| CN110473141A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN104038807B (en) | A kind of figure layer mixed method and device based on OpenGL | |
| CN108364023A (en) | Image-recognizing method based on attention model and system | |
| CN104281864B (en) | A kind of method and apparatus for generating Quick Response Code | |
| CN112651438A (en) | Multi-class image classification method and device, terminal equipment and storage medium | |
| CN106372648A (en) | Multi-feature-fusion-convolutional-neural-network-based plankton image classification method | |
| CN111325271B (en) | Image classification method and device | |
| CN113344806A (en) | Image defogging method and system based on global feature fusion attention network | |
| CN108961253A (en) | A kind of image partition method and device | |
| CN106096668A (en) | The recognition methods of watermarked image and the system of identification | |
| CN104866755B (en) | Setting method and device for background picture of application program unlocking interface and electronic equipment | |
| CN110706151B (en) | Video-oriented non-uniform style migration method | |
| CN106650615A (en) | Image processing method and terminal | |
| CN103714313B (en) | Two-dimensional code identification method and device | |
| CN108898562A (en) | A kind of mobile device image defogging method based on deep learning | |
| CN109816100A (en) | A kind of conspicuousness object detecting method and device based on two-way fusion network | |
| CN109815948A (en) | A kind of paper partitioning algorithm under complex scene | |
| CN109117940A (en) | To accelerated method, apparatus and system before a kind of convolutional neural networks | |
| CN109636764A (en) | A kind of image style transfer method based on deep learning and conspicuousness detection | |
| Luo et al. | An improved resnet algorithm based on cbam | |
| CN112532882A (en) | Image display method and device | |
| CN106681735A (en) | Method, device and apparatus for generating dynamic icons based fonts | |
| CN114359679A (en) | Text recognition method and text recognition system | |
| CN105094615A (en) | Information processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210427 Termination date: 20220209 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |