CN108846404B - Image significance detection method and device based on related constraint graph sorting - Google Patents
Image significance detection method and device based on related constraint graph sorting Download PDFInfo
- Publication number
- CN108846404B CN108846404B CN201810658629.3A CN201810658629A CN108846404B CN 108846404 B CN108846404 B CN 108846404B CN 201810658629 A CN201810658629 A CN 201810658629A CN 108846404 B CN108846404 B CN 108846404B
- Authority
- CN
- China
- Prior art keywords
- node
- value
- ith
- image
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 238000012163 sequencing technique Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 96
- 238000004422 calculation algorithm Methods 0.000 claims description 38
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 7
- 238000009795 derivation Methods 0.000 claims description 5
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 43
- 230000000875 corresponding effect Effects 0.000 description 21
- 230000001276 controlling effect Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000005295 random walk Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image significance detection method and device based on related constraint graph sorting, wherein the method comprises the following steps: performing superpixel segmentation on an image to be detected, establishing a closed-loop graph model, and further calculating prior information of each superpixel node; extracting information such as color, texture, position and the like of an input image; obtaining a foreground probability value of each super-pixel node; taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is larger than the second preset threshold; and calculating by using a model of related constraint graph sequencing to obtain the foreground probability S _ f of each super-pixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final. By applying the embodiment of the invention, the significance detection result can be more accurate.
Description
Technical Field
The invention relates to a method and a device for detecting saliency, in particular to a method and a device for detecting the saliency of an image based on related constraint graph sorting.
Background
With the rapid development of computer and network communication technologies, image data is increasing. The massive multimedia image data provides a great challenge for information processing, and a main research focus in recent years is how to efficiently store, analyze and process the image information. The saliency detection is used as an important preprocessing step for reducing the computational complexity in the field of computer vision, and the saliency target detection is used for positioning and segmenting the most salient foreground target from a scene. The application fields of the technology are particularly wide, such as: target detection and recognition, content-based image retrieval, context-aware based image resizing, video target detection, and the like. How to quickly and accurately find the significant region of the image does not form a complete theoretical system, and has close relation with specific applications, and still remains a challenging subject for researchers.
Currently, visual information processing is generally performed using a bottom-up approach. Bottom-up methods are typically based on underlying visual information, so it is possible to efficiently detect detailed information of the image, rather than global shape information, and the detected salient region may contain only a portion of the object, or be easily mixed with the background. Many bottom-up significance detection models have emerged in recent years: first, Itti et al proposed a neural network-based saliency detection model that combines three feature channels in multiple scales to achieve fast scene analysis, and although the model was able to identify partially salient pixels, the results also included a large number of false positives. Harel et al propose a graph-based saliency detection method, which is a bottom-up model, with final saliency results obtained by calculating dissimilarity. Chang et al constructed a graphical model combining semblance and regional saliency to obtain a better saliency estimate. Wang et al propose a saliency detection model that combines local graph structure and background prior, and an optimized framework, and the final experimental results have good performance in most scenarios. Jiang et al propose the use of an absorbing Markov chain model for saliency detection of images. Tu et al propose to use a minimum spanning tree model for saliency detection of images. Li et al propose to estimate significance values using a regularized random walk ranking model. Yang et al propose a significance detection algorithm based on manifold sorting of the figure, (hereinafter referred to as MR algorithm) this algorithm through screening out some foreground seed points and background seed points, then use the model of manifold sorting to calculate the correlation between these seed points and other nodes, thus get the final significance.
However, the MR algorithm is divided into two stages, the correlations between the remaining nodes and the obtained background seed points are first calculated, a preliminary significant result is obtained after negation is performed, then foreground seed points are obtained on the basis of the first stage, and then the correlations between the remaining nodes and the foreground seed points are calculated, so that a final result is obtained. The two sequencing processes of the method are independently performed, which can cause the technical problem of low accuracy of image significance detection.
Disclosure of Invention
The invention aims to provide an image significance detection method and device based on correlation constraint graph sorting, so as to solve the defects in the traditional manifold sorting model based on graphs.
The invention solves the technical problems through the following technical scheme:
the embodiment of the invention provides an image significance detection method based on related constraint graph sorting, which comprises the following steps:
a: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by taking each non-overlapping superpixel block as a node, and further calculating the central priori information of each node;
b: extracting information such as color, texture, position and the like of an input image;
c: acquiring a foreground probability value of each node by using an MR algorithm;
d: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
e: and calculating by using a model of related constraint graph sequencing to obtain the foreground probability S _ f of each super-pixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final.
Optionally, the step a includes:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed1=(V,E);
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
cicentral prior information of the ith node; x is the number ofiThe abscissa of the central position of the ith node is; y isiIs a longitudinal seat at the central position of the ith nodeMarking; (x)0,y0) The coordinates of the center position of the whole image are shown; sigma1The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Optionally, the undirected edge is obtained by:
for the neighboring node l of the two neighbors k of each node i, the formula is used, dist (k, l) | | xk-xl||2Calculating the Euclidean distance dist (k, l) between the color and k; if the Euclidean distance of the color is smaller than the threshold value theta, connecting a non-directional edge between the node l and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein dist (k, l) is the Euclidean distance of the color between the ith node and the kth node; x is the number ofkIs the color value of the kth node; x is the number oflIs the color value of the l node; and | | is a modulo function.
Optionally, step B includes: according to the color, texture and position information of the extracted image; by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein,
a weight of an undirected edge between the ith node and the jth node; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. ofiIs a feature descriptor of the ith node, and vi∈R65,vi=[xi,yi,Li,ai,bi,ci,ωi];(xi,yi) Representing the center position coordinates of each superpixel node; (L)i,ai,bi) Expressed is the inclusion of each super-pixel node in the CIE LAB color spaceThe color mean value of all contained pixel points; c. CiCentral prior information of the ith node; omegaiIs the LBP value of the ith node; v. ofjA feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.
Optionally, step C includes:
c1: acquiring the weight of each undirected edge of each node in the MR algorithm;
c2: constructing a second incidence matrix of the MR algorithm according to the weight of each non-directional edgeWherein,
is the weight of the edge between the ith and jth superpixels, andW2is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. CiThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. CjThe color mean value of all pixel points of the jth node in the CIELAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D11,…,dnn-calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; diiIs a degree matrix element, and the weight of the non-directional edge corresponding to the second incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → RmCalculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f1,…,fn]T;f1Is the ranking value of the 1 st node; f. ofnIs the ranking value of the nth node; n is the number of nodes; let y be [ y1,y2,…yn]TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; rmIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. ofiIs the ranking value of the ith node; f. ofjIs the ranking value of the jth node; y isiIs the label value of the ith node;a weight of the undirected edge; diiThe element of the ith row and the ith column in the degree matrix; djjThe element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a non-normalized solution, wherein,
d is a degree matrix; w2Is a second incidence matrix; s is W2The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
Optionally, the step D includes:
d1: by means of the formula (I) and (II),acquiring a first preset threshold and a second preset threshold; wherein,
h1is a first preset threshold value; h is2Is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;
d2: by means of the formula (I) and (II),obtaining a foreground seed point set ind _ form and a background seed point set ind _ back, wherein,
ind _ form is a foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.
Optionally, the step E includes:
e1: using the formula, F: X → RnCalculating the corresponding sorting weight of the image to be detected, wherein,
f is a sorting function; fiDenoted is the rank value of the ith node, and F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background;
e2: by means of the formula (I) and (II),acquiring a label value of each node; using the formula, Y ═ Y1,y2)∈Rm×2And obtaining a label vector of each node, wherein,
y is a label vector of each node; y is1A label value for which the node belongs to the foreground; y is2A label value for which the node belongs to a background; rm×2Is a real space with 2m dimension;
e3: the model formula of the constructed related constraint graph ordering is as follows,
F*is a closed solution; wijA first incidence matrix corresponding to the weight of the undirected edge between the ith node and the jth node; fiIs the ranking value of the ith node; fjIs the ranking value of the jth node; diIs a degree matrix element; f. ofiForeground probability of the ith node; giThe background probability of the ith node; w is aiThe characteristic weight of the ith node; x is the number ofiIs the characteristic of the ith node; biIs a bias parameter; beta is a1Linear constraint coefficients to the foreground probability; beta is a2Linear constraint coefficient to background probability;
e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.
The embodiment of the invention also provides an image significance detection device based on the related constraint graph sorting, which comprises the following steps:
the first calculation module is used for carrying out superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm aiming at each image to be detected to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by taking each non-overlapping superpixel block as a node, and further calculating the central prior information of each node;
the input module is used for extracting information such as color, texture, position and the like of an input image;
the second calculation module is used for acquiring the foreground probability value of each node by using the MR algorithm;
the first setting module is used for taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
and the second setting module is used for calculating the foreground probability S _ f and the background probability S _ g of each super-pixel node by using the model ordered by the related constraint graph, and using the foreground probability value S _ f as a final significant estimation value S _ final.
Optionally, the first computing module is further configured to:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed1=(V,E);
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
cicentral prior information of the ith node; x is the number ofiThe abscissa of the central position of the ith node is; y isiIs the longitudinal coordinate of the central position of the ith node; (x)0,y0) The coordinates of the center position of the whole image are shown; sigma1The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Optionally, the second calculating module is further configured to:
c1: acquiring the weight of each undirected edge of each node in the MR algorithm;
C2:constructing a second incidence matrix of the MR algorithm according to the weight of each non-directional edgeWherein,
is the weight of the edge between the ith and jth superpixels, andW2is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. CiThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. CjThe color mean value of all pixel points of the jth node in the CIELAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D11,…,dnn-calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; diiIs a degree matrix element, and weights of the non-directional edges corresponding to the incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → RmCalculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f1,…,fn]T;f1Is the ranking value of the ith node; f. ofnIs the ranking value of the nth node; n is the number of nodes; let y be [ y1,y2,…yn]TRepresenting the label vector, speciesThe label value of the child point is 1, and the label values of the other nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; rmIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. ofiIs the ranking value of the ith node; f. ofjIs the ranking value of the jth node; y isiIs the label value of the ith node;a weight of the undirected edge; diiThe element of the ith row and the ith column in the degree matrix; djjThe element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a non-normalized solution, wherein,
d is a degree matrix; w2Is the second incidence matrix; s is W2The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
Optionally, the second calculating module is further configured to:
according to the color, texture and position information of the extracted image;
by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein,
a weight of an undirected edge between the ith node and the jth node; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. ofiIs a feature descriptor of the ith node, and vi∈R65,vi=[xi,yi,Li,ai,bi,ci,ωi];(xi,yi) Representing the center position coordinates of each superpixel node; (L)i,ai,bi) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. CiCentral prior information of the ith node; omegaiIs the LBP value of the ith node; v. ofjA feature descriptor for the jth node; σ is a preset constant for controlling the weight balance.
Compared with the prior art, the invention has the following advantages:
by applying the embodiment of the invention, the association parameters between the foreground clues and the background clues are introduced when the graph ranking function is constructed, and compared with the prior art in which the association between the foreground clues and the background clues is not considered when the graph ranking function is constructed, more influence factors are considered, and the significance detection result is more accurate. Meanwhile, because the traditional mode based on composition ignores the effect of image features when the significance calculation is carried out, the final significance value is restrained by utilizing the linear learning of the image features, the composition information and the feature information are fully utilized, and the final detection result is further improved.
Drawings
Fig. 1 is a schematic flowchart of an image saliency detection method based on a related constraint map sorting according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a method for detecting image saliency based on relevance constraint map ranking according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a constructed closed-loop graph model according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an image saliency detection apparatus based on a related constraint map sorting according to an embodiment of the present invention.
Detailed Description
The following examples are given for the detailed implementation and specific operation of the present invention, but the scope of the present invention is not limited to the following examples.
In order to solve the problems in the prior art, embodiments of the present invention provide a method and an apparatus for detecting image saliency based on a correlation constraint graph sorting.
Fig. 1 is a schematic flowchart of an image saliency detection method based on a related constraint map sorting according to an embodiment of the present invention; fig. 2 is a schematic diagram illustrating a principle of an image saliency detection method based on a related constraint map sorting according to an embodiment of the present invention, as shown in fig. 1 and fig. 2, the method includes:
s101: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by using each non-overlapping superpixel block as a node, and further calculating prior information of each node;
specifically, the step S101 may include: a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then obtaining the non-directional edge corresponding to each node, wherein each node has an edge with the node directly adjacent to the node and the adjacent (two adjacent) node, calculating to obtain the connected edge aiming at the two adjacent nodes of each node by using a formula, and connecting the nodes on the four boundaries, thereby constructing the non-directional graph model G1(V, E); a2: by means of the formula (I) and (II),calculating central prior information of each node, wherein ciCentral prior information of the ith node; x is the number ofiThe abscissa of the central position of the ith node is; y isiIs the longitudinal coordinate of the central position of the ith node; (x)0,y0) The coordinates of the center position of the whole image are shown; sigma1The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Fig. 3 is a schematic structural diagram of a constructed closed-loop graph model according to an embodiment of the present invention, and as shown in fig. 3, the construction process of fig. 3 is: carrying out superpixel segmentation on an image to be detected to obtain N superpixel blocks, taking each superpixel block as a node, connecting each superpixel with adjacent superpixels in a local region, constructing edges, and finally establishing a closed-loop graph model G1。
In practical application, the connection situation of the edge between each node can be divided into the following four situations:
1. there is an edge between each node i and its immediately adjacent node j.
2. Each node i has an edge with its two neighbors (neighbors) node k.
3. The neighboring node l of the two neighbors k of each node i uses the formula, dist (k, l) | | xk-xl||2Calculating the color Euclidean distance dist (k, l) between the node and the k, if the distance is smaller than a threshold value theta, considering that an edge is connected between the node and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein the dist (k, l) is the color Euclidean distance between the ith node and the kth node; x is the number ofkIs the color value of the kth node; x is the number oflIs the color value of the l node; and | | is a modulo function.
4. And all nodes positioned on the four boundaries are connected with each other to form closed-loop connection around the image.
By applying the embodiment of the invention, compared with the closed-loop graph model in the prior art, the 3 rd point is added into the constructed closed-loop graph model, the local smooth range of each super-pixel region is enlarged, the consistent characteristics among the super-pixel regions in certain adjacent regions are better explained, and the accuracy of image significance detection is further improved.
S102: information such as color, texture, position, etc. of the input image is extracted.
Specifically, the color, texture, and position information of the extracted image may be used;
then the formula is utilized to obtain the final product,calculating the weight of each undirected edge to construct a first incidence matrixWherein,
a weight of an undirected edge between the ith node and the jth node; w1Is a first incidence matrix; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. ofiBeing the i-th nodeFeature descriptors, and vi∈R65,vi=[xi,yi,Li,ai,bi,ci,ωi];(xi,yi) Representing the center position coordinates of each superpixel node; (L)i,ai,bi) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. CiCentral prior information of the ith node; omegaiIs the LBP value of the ith node; v. ofjA feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.
As shown in fig. 2, in practical applications, the color feature of the image is a CIE LAB (LAB specified by the organization of the Commission International Eclairage light standard) color mean value that each super-pixel region includes a pixel point; the texture features are Local Binary Patterns (LBP) features. Computing the weight W of an edge between two connected superpixel nodes by computing the difference in feature combinations between them2。
S103: and acquiring the foreground probability value of each node by using an MR algorithm.
Specifically, the step S103 may include:
c1: performing superpixel segmentation on an image to be detected by using a SLIC algorithm, and obtaining n non-overlapping superpixel block sets X ═ X by segmentation1,…xq,xq+1,…xn-wherein the first q superpixel blocks are marked query seed points, and the rest are unmarked superpixel nodes. Then, a closed-loop graph model G is constructed2(V, E). V represents all node sets, and E represents all undirected edge sets. Wherein, each node and its direct adjacent node, and adjacent node have the edge, the node on four borders interconnect.
C2: constructing a second incidence matrixWherein,is the weight of the edge between the ith and jth superpixels, andW2is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. CiThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. CjThe color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; σ is a constant that controls the balance of weights, and is usually positively correlated with the Euclidean distance of the color mean between two nodes.
C3: according to the formula, D ═ diag { D ═ D11,…,dnnCalculating a degree matrix, wherein D is the degree matrix; diag { } is a diagonal matrix construction function; diiIs a degree matrix element, and and the weights of the non-directional edges corresponding to the incidence matrix.
C4: for each node on the boundary, the labeled value of the node is labeled according to the boundary prior.
C5: using the formula, f: X → RmAnd calculating the sorting weight corresponding to the image to be detected, wherein f is a sorting function, and f is [ f ═ f [ ]1,…,fn]T;f1Is the ranking value of the 1 st node; f. ofnIs the ranking value of the nth node; n is the number of nodes; let y be [ y1,y2,…yn]TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; rmIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: using a ranking functionThe formula is shown in the figure,computing a closed solution, wherein f*Is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. ofiIs the ranking value of the ith node; f. ofjIs the ranking value of the jth node; y is1Is the label value of the ith node;a weight of the undirected edge; diiThe element of the ith row and the ith column in the degree matrix; djjThe element of j row and j column in the degree matrix; μ is the equilibrium parameter.
C7: using a formula in accordance with the closed solution,obtaining a non-normalized solution, wherein D is a degree matrix; w2Is the second incidence matrix; s is W2The normalization matrix of (2).
The closed solution calculated in the step C6 may be:
C8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain the background probability value f of each node under four conditions, wherein the background probability value f is the value of the background probability of each node under four conditionsAnd λ is a preset parameter.
In practical application, in image saliency detection, a degree matrix is usually used to replace an identity matrix in a closed solution, so as to obtain the above formula. For example, the correlation value between each node and the background seed points on the four boundaries, i.e., the background probability value of each node in the four cases, is calculated as f.
C9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing dot multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value.
In practical application, the value obtained by inverting the normalized background probability value is as follows:
the point multiplication of the significant values obtained in the four cases to obtain an initial result may be:
Sbq(i)=St(i)×Sb(i)×Sl(i)×Sr(i)。
it is emphasized that in step S103, the foreground probability value of each node is obtained by using an MR algorithm, and S _ MR (foreground probability value) of the image to be detected, which is obtained in the first stage of the classical graph-based manifold sorting algorithm, is used.
Taking each superpixel as a node, connecting each superpixel with the superpixel which belongs to the adjacent superpixel in the local area, constructing edges, and then establishing a closed-loop graph model G2. Then extracting LAB color characteristics of each super-pixel region, and calculating the weight W of the edge by calculating the difference value on the color characteristics between two connected super-pixel nodes2. The algorithm is mainly divided into two stages, and the supergrams on four boundaries of the image are selected according to the prior information calculated in the step S101The pixels are used as background Query points, then the correlation between each super-pixel node and the background Query points is calculated by using a manifold sorting algorithm, the probability that each super-pixel belongs to the background is calculated, then normalization is carried out, and then negation is carried out, so that an initial significant result is obtained. In the second stage, firstly, binarization is carried out on the initial result of the first stage, then foreground Query points are screened out, and then the correlation between each super-pixel node and each foreground Query point is calculated by using a manifold sorting algorithm, so that the S _ MR (foreground probability value) of the image to be detected is obtained.
S104: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold.
Specifically, the step S104 may include:
d1: by means of the formula (I) and (II),acquiring a first preset threshold and a second preset threshold; h is1Is a first preset threshold value; h is2Is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;
d2: by means of the formula (I) and (II),acquiring a foreground seed point set ind _ form and a background seed point set ind _ back, wherein the ind _ form is the foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.
S105: and calculating by using a model of related constraint graph sequencing to obtain the foreground probability S _ f of each super-pixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final.
Specifically, the step S105 may include: e1: using the formula, F: X → RnCalculating the corresponding sorting weight of the image to be detected, wherein F is a sorting function; fiIndicating the ranking value of the ith nodeAnd F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background; e2: by means of the formula (I) and (II),acquiring a label value of each node; using the formula, Y ═ Y1,y2)∈Rm×2And obtaining a label vector of each node, wherein,
y is a label vector of each node; y is1A label value for which the node belongs to the foreground; y is2A label value for which the node belongs to a background; rm×2Is a real space of 2m dimensions.
E3: the model formula of the constructed related constraint graph ordering is as follows,
wherein, F*Is a closed solution; wijA first incidence matrix corresponding to the weight of the undirected edge between the ith node and the jth node; fiIs the ranking value of the ith node; fjIs the ranking value of the jth node; diIs a degree matrix element; f. ofiForeground probability of the ith node; giThe background probability of the ith node; w is aiThe characteristic weight of the ith node; x is the number ofiIs the characteristic of the ith node; biIs a bias parameter; beta is a1Linear constraint coefficients to the foreground probability; beta is a2Linear constraint coefficient to background probability; e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.
In the formula in step E3, the first polynomial is a smoothing term, and since the surrounding area of the area with a certain characteristic in the image also has similar features to it, i.e. the scores of the rankings between the nodes in the adjacent local areas are considered as similar as possible, the smoothing term can be added; the second term is a fitting term, so that the difference between the ranking value we finally calculate and the initial label value we give is as small as possible; the third term is a constraint term for f and g, in order to make the correlation between f and g calculated as small as possible; the fourth and fifth terms are linear constraint terms for f and g respectively, and the final significant value is constrained by linear learning of image features.
In practical applications, the step of E4 may include:
1) simplifying the model formula of the constructed related constraint graph sequencing to obtain an optimized formula,
2) fixing f can obtain the optimal solution of b and W, so there are:
3) writing the solving problem of the model with the sequencing of the related constraint graphs into the following formula:
J=Tr[FTA*F-μFTY]+λfTg+β1||XTWf+bf1-f||2+β2||XTWg+bg1-g||2wherein
A*=(1+μ)D-W。
4) replacing F in the formula with FIn the alternative, F is (F, g), and Y is (Y)1,y2) And solving the reduction can obtain a formula,
5) and f in the step 4) is subjected to derivation to obtain the following results:
6) and g in the step 4) is subjected to derivation to obtain the following results:
7) from 5) and 6) can be obtained:
8) calculated according to the formula of 7):
f*=μ(λ2I-4(A*)2-2A*β1B-2β2BA*-β2β1B2)-1(λy2-2A*y1-β2By1);
will f is*As the final significant estimate S final.
By applying the embodiment shown in the figure 1 of the invention, when the graph ranking function is constructed, the association parameters between the foreground clues and the background clues are introduced, and when the association between each super-pixel node and the given foreground query point and the background query is calculated at the same time, a correlation constraint condition is added, so that the correlation between the obtained foreground probability value and the obtained background probability value is reduced. Meanwhile, because the traditional mode based on composition ignores the effect of image features when the significance calculation is carried out, the final significance value is restrained by utilizing the linear learning of the image features, the composition information and the feature information are fully utilized, and the final detection result is further improved.
Corresponding to the image significance detection method based on the related constraint graph sorting provided by the embodiment of the invention, the embodiment of the invention also provides an image significance detection device based on the related constraint graph sorting.
Fig. 4 is a schematic structural diagram of an image saliency detection apparatus based on a related constraint map sorting according to an embodiment of the present invention, as shown in fig. 4, the apparatus includes:
the first calculation module 401 is configured to perform superpixel segmentation on each image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain non-overlapping superpixel blocks, then establish a closed-loop graph model by using each non-overlapping superpixel block as a node, and further calculate prior information of each node;
an input module 402, configured to extract information such as color, texture, and position of an input image;
a second calculating module 403, configured to obtain a foreground probability value of each node by using an MR algorithm;
a first setting module 404, configured to use a set of nodes with a foreground probability value greater than a first preset threshold as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
and a second setting module 405, configured to calculate a foreground probability S _ f and a background probability S _ g of each super-pixel node by using the model ordered by the relevant constraint graph, and use the foreground probability value S _ f as a final significant estimation value S _ final.
By applying the embodiment shown in fig. 4 of the invention, when the graph ranking function is constructed, the association parameters between the foreground clues and the background clues are introduced, and when the association between each super-pixel node and the given foreground query point and the background query is calculated at the same time, a correlation constraint condition is added, so that the correlation between the obtained foreground probability value and the obtained background probability value is reduced. Meanwhile, because the traditional mode based on composition ignores the effect of image features when the significance calculation is carried out, the final significance value is restrained by utilizing the linear learning of the image features, the composition information and the feature information are fully utilized, and the final detection result is further improved.
In a specific implementation manner of the embodiment of the present invention, the first calculating module 404 is further configured to:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed1=(V,E);
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
cicentral prior information of the ith node; x is the number ofiThe abscissa of the central position of the ith node is; y isiIs the longitudinal coordinate of the central position of the ith node; (x)0,y0) The coordinates of the center position of the whole image are shown; sigma1The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
In a specific implementation manner of the embodiment of the present invention, the second calculating module 403 is further configured to:
c1: acquiring the weight of each undirected edge of each node in the MR algorithm;
c2: constructing a second incidence matrix of the MR algorithm according to the weight of each non-directional edgeWherein,
is the weight of the edge between the ith and jth superpixels, andW2is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. CiThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. CjThe color mean value of all pixel points of the jth node in the CIELAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D11,…,dnn-calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; diiIs a degree matrix element, and weights of the non-directional edges corresponding to the incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → RmCalculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f1,…,fn]T;f1Is the ranking value of the 1 st node; f. ofnIs the ranking value of the nth node; n is the number of nodes; let y be [ y1,y2,…yn]TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; rmIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f*is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. ofiIs the ranking value of the ith node; f. ofjIs the ranking value of the jth node; y isiIs the label value of the ith node;a weight of the undirected edge; diiThe element of the ith row and the ith column in the degree matrix; djjThe element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a non-normalized solution, wherein D is a degree matrix; w2Is the second incidence matrix; s is W2The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain the background probability value f of each node under four conditions, wherein lambda is a preset parameter;
C9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
In a specific implementation manner of the embodiment of the present invention, the second calculating module 403 is further configured to:
by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein,
a weight of an undirected edge between the ith node and the jth node; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. ofiIs a feature descriptor of the ith node, and vi∈R65,vi=[xi,yi,Li,ai,bi,ci,ωi];(xi,yi) Representing the center position coordinates of each superpixel node; (L)i,ai,bi) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. CiCentral prior information of the ith node; omegaiIs the LBP value of the ith node; v. ofjA feature descriptor for the jth node; σ is a preset constant for controlling the weight balance.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. An image saliency detection method based on relevance constraint graph sorting is characterized by comprising the following steps:
a: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by taking each non-overlapping superpixel block as a node, and further calculating the central priori information of each node;
b: extracting color, texture and position information of an input image;
c: obtaining a foreground probability value of each node by using a significance detection algorithm based on manifold sorting of a graph;
d: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
e: calculating by using a model of related constraint graph sequencing to obtain a foreground probability S _ f of each super pixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final;
and step C, comprising:
c1: acquiring the weight of each undirected edge of each node in a significance detection algorithm based on manifold sorting of a graph;
c2: constructing a second incidence matrix of the significance detection algorithm based on manifold sorting of the graph according to the weight of each undirected edgeWherein,
is the weight of the edge between the ith super pixel and the jth super pixel,and isW2Is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. CiThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. CjThe color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D11,…,dnn-calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; diiIs a degree matrix element, and weights of the non-directional edges corresponding to the incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → RmCalculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f1,…,fn]T;f1Is the ranking value of the 1 st node; f. ofnIs the ranking value of the nth node; n is the number of nodes; let y be [ y1,y2,…yn]TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; rmIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f*is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. ofiIs the ranking value of the ith node; f. ofjIs the ranking value of the jth node; y isiIs the label value of the ith node;a weight of the undirected edge; diiThe element of the ith row and the ith column in the degree matrix; djjThe element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a non-normalized solution, wherein,
d is a degree matrix; w2Is the second incidence matrix; s is W2The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
2. The method for detecting the significance of the image based on the related constraint map sorting as claimed in claim 1, wherein the step A comprises:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed1=(V,E);
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
cicentral prior information of the ith node; x is the number ofiThe abscissa of the central position of the ith node is; y isiIs the longitudinal coordinate of the central position of the ith node; (x)0,y0) The coordinates of the center position of the whole image are shown; sigma1The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
3. The method for detecting the saliency of images based on the ordering of related constraint maps according to claim 2, characterized in that the undirected edges are obtained by:
for the neighboring node l of the two neighbors k of each node i, the formula is used, dist (k, l) | | xk-xl||2Calculating the Euclidean distance between the color of the color k and the color k; if the Euclidean distance of the color is smaller than the threshold value theta, connecting a non-directional edge between the node l and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein dist (k, l) is the Euclidean distance of the color between the ith node and the kth node; x is the number ofkIs the color value of the kth node; x is the number oflIs the color value of the l node; and | | is a modulo function.
4. The method for detecting the saliency of images based on the ordering of correlation constraint maps according to claim 1, wherein the step B comprises:
according to the color, texture and position information of the extracted image; by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein,
a weight of an undirected edge between the ith node and the jth node; w1Is a first incidence matrix; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. ofiIs a feature descriptor of the ith node, and vi∈R65,vi=[xi,yi,Li,ai,bi,ci,ωi];(xi,yi) Representing the center position coordinates of each superpixel node; (L)i,ai,bi) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. CiCentral prior information of the ith node; omegaiIs the LBP value of the ith node; v. ofjA feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.
5. The method for detecting the significance of the image based on the related constraint map sorting as claimed in claim 1, wherein the step D comprises:
d1: by means of the formula (I) and (II),acquiring a first preset threshold and a second preset threshold; wherein,
h1is a first preset threshold value;h2is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;
d2: by means of the formula (I) and (II),obtaining a foreground seed point set ind _ form and a background seed point set ind _ back, wherein,
ind _ form is a foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.
6. The method for detecting the significance of the image based on the related constraint map ordering according to claim 1, wherein the step E comprises the following steps:
e1: using the formula, F: X → RnCalculating the corresponding sorting weight of the image to be detected, wherein,
f is a sorting function; fiDenoted is the rank value of the ith node, and F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background;
e2: by means of the formula (I) and (II),acquiring a label value of each node; using the formula, Y ═ Y1,y2)∈Rm×2And obtaining a label vector of each node, wherein,
y is a label vector of each node; y is1A label value for which the node belongs to the foreground; y is2A label value for which the node belongs to a background; rm×2Is a real space with 2m dimension;
e3: the model formula of the constructed related constraint graph ordering is as follows,
F*is a closed solution; wijAs a non-directional edge between the ith node and the jth nodeA first incidence matrix corresponding to the weight of (a); fiIs the ranking value of the ith node; fjIs the ranking value of the jth node; diIs a degree matrix element; f. ofiForeground probability of the ith node; giThe background probability of the ith node; w is aiThe characteristic weight of the ith node; x is the number ofiIs the characteristic of the ith node; biIs a bias parameter; beta is a1Linear constraint coefficients to the foreground probability; beta is a2Linear constraint coefficient to background probability;
e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.
7. An apparatus for detecting image saliency based on relevance constraint map ordering, the apparatus comprising:
the first calculation module is used for carrying out superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm aiming at each image to be detected to obtain non-overlapping superpixel blocks, then establishing a closed-loop graph model by taking each non-overlapping superpixel block as a node, and further calculating the central prior information of each node;
the input module is used for extracting color, texture and position information of an input image;
the second calculation module is used for acquiring the foreground probability value of each node by utilizing a significance detection algorithm based on manifold sorting of the graph;
the first setting module is used for taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
the second setting module is used for calculating a foreground probability S _ f and a background probability S _ g of each super-pixel node by using a model ordered by a related constraint graph, and using the foreground probability value S _ f as a final significant estimation value S _ final;
the second computing module is further configured to:
c1: acquiring the weight of each undirected edge of each node in a significance detection algorithm based on manifold sorting of a graph;
c2: constructing a second incidence matrix of the significance detection algorithm based on manifold sorting of the graph according to the weight of each undirected edgeWherein,
is the weight of the edge between the ith and jth superpixels, andW2is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. CiThe color mean value of all pixel points of the ith node in the CIE LAB color space; c. CjThe color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D11,…,dnn-calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; diiIs a degree matrix element, and the weight of the non-directional edge corresponding to the second incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → RmCalculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f1,…,fn]T;f1Is the ranking value of the 1 st node; f. ofnIs the ranking value of the nth node; n is the number of nodes; let y be [ y1,y2,…yn]TRepresenting a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; rmIs m-dimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f*is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. ofiIs the ranking value of the ith node; f. ofjIs the ranking value of the jth node; y isiIs the label value of the ith node; y isnThe weight of the nth node;a weight of the undirected edge; diiThe element of the ith row and the ith column in the degree matrix; djjThe element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a non-normalized solution, wherein,
d is a degree matrix; w2Is the second incidence matrix; s is W2The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
8. The apparatus according to claim 7, wherein the first computing module is further configured to:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a non-directional edge corresponding to each node is obtained, and a non-directional graph model G is further constructed1=(V,E);
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
cicentral prior information of the ith node; x is the number ofiThe abscissa of the central position of the ith node is; y isiIs the longitudinal coordinate of the central position of the ith node; (x)0,y0) The coordinates of the center position of the whole image are shown; sigma1The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810658629.3A CN108846404B (en) | 2018-06-25 | 2018-06-25 | Image significance detection method and device based on related constraint graph sorting |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810658629.3A CN108846404B (en) | 2018-06-25 | 2018-06-25 | Image significance detection method and device based on related constraint graph sorting |
Publications (2)
Publication Number | Publication Date |
---|---|
CN108846404A CN108846404A (en) | 2018-11-20 |
CN108846404B true CN108846404B (en) | 2021-10-01 |
Family
ID=64203559
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810658629.3A Active CN108846404B (en) | 2018-06-25 | 2018-06-25 | Image significance detection method and device based on related constraint graph sorting |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108846404B (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109993173B (en) * | 2019-03-28 | 2023-07-21 | 华南理工大学 | Weak supervision image semantic segmentation method based on seed growth and boundary constraint |
CN110111353B (en) * | 2019-04-29 | 2020-01-24 | 河海大学 | Image significance detection method based on Markov background and foreground absorption chain |
CN110188763B (en) * | 2019-05-28 | 2021-04-30 | 江南大学 | Image significance detection method based on improved graph model |
CN110287802B (en) * | 2019-05-29 | 2022-08-12 | 南京邮电大学 | Human eye gaze point prediction method based on optimized image foreground and background seeds |
CN110298842A (en) * | 2019-06-10 | 2019-10-01 | 上海工程技术大学 | A kind of rail clip image position method based on super-pixel node sequencing |
CN110533593B (en) * | 2019-09-27 | 2023-04-11 | 山东工商学院 | Method for quickly creating accurate trimap |
CN117372431B (en) * | 2023-12-07 | 2024-02-20 | 青岛天仁微纳科技有限责任公司 | Image detection method of nano-imprint mold |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6038344A (en) * | 1996-07-12 | 2000-03-14 | The United States Of America As Represented By The Secretary Of The Navy | Intelligent hypersensor processing system (IHPS) |
CN104123734A (en) * | 2014-07-22 | 2014-10-29 | 西北工业大学 | Visible light and infrared detection result integration based moving target detection method |
CN104715251A (en) * | 2015-02-13 | 2015-06-17 | 河南科技大学 | Salient object detection method based on histogram linear fitting |
-
2018
- 2018-06-25 CN CN201810658629.3A patent/CN108846404B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6038344A (en) * | 1996-07-12 | 2000-03-14 | The United States Of America As Represented By The Secretary Of The Navy | Intelligent hypersensor processing system (IHPS) |
CN104123734A (en) * | 2014-07-22 | 2014-10-29 | 西北工业大学 | Visible light and infrared detection result integration based moving target detection method |
CN104715251A (en) * | 2015-02-13 | 2015-06-17 | 河南科技大学 | Salient object detection method based on histogram linear fitting |
Also Published As
Publication number | Publication date |
---|---|
CN108846404A (en) | 2018-11-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108846404B (en) | Image significance detection method and device based on related constraint graph sorting | |
CN107066559B (en) | Three-dimensional model retrieval method based on deep learning | |
CN107291945B (en) | High-precision clothing image retrieval method and system based on visual attention model | |
Li et al. | Robust saliency detection via regularized random walks ranking | |
CN112184752A (en) | Video target tracking method based on pyramid convolution | |
CN110569738B (en) | Natural scene text detection method, equipment and medium based on densely connected network | |
CN111881714A (en) | Unsupervised cross-domain pedestrian re-identification method | |
CN112101150A (en) | Multi-feature fusion pedestrian re-identification method based on orientation constraint | |
CN111625667A (en) | Three-dimensional model cross-domain retrieval method and system based on complex background image | |
CN109086777B (en) | Saliency map refining method based on global pixel characteristics | |
CN102982539B (en) | Characteristic self-adaption image common segmentation method based on image complexity | |
CN112529005B (en) | Target detection method based on semantic feature consistency supervision pyramid network | |
CN111126385A (en) | Deep learning intelligent identification method for deformable living body small target | |
CN112712546A (en) | Target tracking method based on twin neural network | |
CN107862680B (en) | Target tracking optimization method based on correlation filter | |
CN111914642A (en) | Pedestrian re-identification method, device, equipment and medium | |
CN104732534B (en) | Well-marked target takes method and system in a kind of image | |
CN111768415A (en) | Image instance segmentation method without quantization pooling | |
CN110598715A (en) | Image recognition method and device, computer equipment and readable storage medium | |
CN110765882A (en) | Video tag determination method, device, server and storage medium | |
CN110827312A (en) | Learning method based on cooperative visual attention neural network | |
CN112287935B (en) | Image semantic segmentation method and system based on significance prior | |
CN112288761A (en) | Abnormal heating power equipment detection method and device and readable storage medium | |
CN111241924A (en) | Face detection and alignment method and device based on scale estimation and storage medium | |
CN112132145A (en) | Image classification method and system based on model extended convolutional neural network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |