CN109118493B - Method for detecting salient region in depth image - Google Patents
Method for detecting salient region in depth image Download PDFInfo
- Publication number
- CN109118493B CN109118493B CN201810757983.1A CN201810757983A CN109118493B CN 109118493 B CN109118493 B CN 109118493B CN 201810757983 A CN201810757983 A CN 201810757983A CN 109118493 B CN109118493 B CN 109118493B
- Authority
- CN
- China
- Prior art keywords
- value
- pixel
- depth
- significance
- depth image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for detecting a salient region in a depth image. Firstly, calculating the difference between each pixel point and adjacent points in a depth map to obtain the gradient characteristic of each pixel point; then, an initial saliency map is obtained by gradient feature calculation by adopting a global contrast method; then, carrying out peak-valley detection and zero-parallax region estimation according to the histogram statistical characteristics of the depth map, and implementing division of a background region and a foreground region, thereby inhibiting the significance of a background part and keeping the significance of the foreground part; and finally, further optimizing by using the extended super-pixel division method to obtain a final salient region detection image. The method effectively inhibits the significance of the background part and optimizes the significance of the foreground part by dividing the background area and the superpixel, provides reliable significant areas for target detection, target identification, scene understanding and the like, and improves the acquisition capability of the image interesting area.
Description
Technical Field
The invention relates to a region detection method technology, in particular to a method for detecting a salient region in a depth image.
Background
Depth information is one of important information channels of human visual attention mechanisms, and can help people to quickly find interested areas in complex scenes.
With the progress of sensor technology, the acquisition of depth images is easier, and how to understand a three-dimensional scene represented by the depth images is an important problem to be solved in the fields of intelligent robot navigation, environment modeling, somatosensory games and the like. The salient regions may be used to guide the robot in discovering potential objects in the scene and reduce the computational load of environmental understanding.
The currently common method for calculating the significant area of the depth map is based on the global contrast of the depth value, the depth value is directly used for calculation, and the method is easily interfered by noise data in the depth map, so that the detection result is inaccurate. Furthermore, a change in the depth value range of the different depth map makes the detection result unstable.
Disclosure of Invention
The invention aims to provide a method for detecting a salient region of a depth image, which can accurately and stably estimate the salient region.
The technical solution for realizing the purpose of the invention is as follows: a salient region detection method of a depth image comprises the following steps:
Step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation mode according to the gradient features extracted in the step 1k) Obtaining an initial saliency map of the same resolution;
step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs;
step 4, estimating a zero parallax area ZPA of the depth image;
step 5, dividing a background area and a foreground area in the depth image according to the peak and trough detection result and the zero parallax area ZPA, adjusting the saliency map obtained in the step 2 according to the background area and the foreground area, and inhibiting the saliency value of the background area to obtain an improved saliency map;
and 6, performing superpixel segmentation on the original image by adopting a superpixel segmentation algorithm, and then optimizing the saliency map obtained in the step 5 to obtain a final saliency area.
Compared with the prior art, the invention has the following remarkable advantages: (1) the method adopts the gradient characteristic of the depth value as the calculation basis of the overall contrast, is less interfered by noise and is not influenced by the change of the range of the depth value, so that the result is more accurate; (2) the invention effectively inhibits the significance of the background and the salient target area by utilizing the method of dividing the background area, and improves the stability of the detection result.
The invention is further described below with reference to the accompanying drawings.
Drawings
Fig. 1 is a flowchart of a salient region detection method of a depth map according to the present invention.
Fig. 2 is a schematic diagram of salient region detection according to an embodiment of the present invention, in which a diagram (a) is an original image, a diagram (b) is a depth image, and a diagram (c) is a detection result diagram.
Detailed Description
A method for detecting a salient region in a depth image comprises the following steps:
In a further embodiment, extracting the gradient feature of the depth map specifically includes the following steps:
step 1-1, traversing all pixel points of the depth image I to obtain a gradient vector of each pixel point, and counting the pixel points IkN, N is the total number of pixels, and its gradient vector (dr) isk,dck) The calculation formula is as follows:
drk=(dep(r+1,c)-dep(r-1,c)/2 (1)
dck=(dep(r,c+1)-dep(r,c-1)/2 (2)
wherein r and c correspond to rows and columns of image coordinates, dep (r, c) represents the depth value of the r-th row and c-th column in the depth image I;
step 1-2, traversing all pixel points to obtain the gradient characteristic of each point, and obtaining a pixel point IkCharacteristic of gradient ofThe method specifically comprises the following steps:
Step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation mode according to the gradient features extracted in the step 1k) Obtaining an initial saliency map of the same resolution;
in a further embodiment, an initial saliency value S (I) of each pixel is calculated by a global contrast method according to the gradient characteristicsk) Obtaining an initial saliency map of the same resolution, specifically comprising the following steps:
step 2-1, normalizing two elements of all gradient feature vectors to an interval [0, 255%]Rounding off to obtain integer value and pixel point IkAfter being normalized, the gradient features ofThereby the gradient characteristic values of all pixel pointsAll correspond to [0,255]An integer of up to 256 different values, notedThe same can be obtained
Step 2-2, obtaining characteristic values according to the calculation mode of the global contrastCorresponding significance values:
where n is 256, the total number of feature values extracted from the depth image, and fjRepresentsThe probability of occurrence in the image is,is composed ofAnda distance metric function of the two features; for characteristic valueIn the same way, the corresponding significance values are obtained:
step 2-3, the corresponding significance values of the pixels with the same characteristic value are also the same, and the pixel I is subjected tokIf its characteristic value isThe initial saliency value for that pixel is then:
wherein, waAnd wbIs a weight parameter; for each pixel, according to the characteristic value of the pixel, the significance value of the pixel can be obtained, and therefore the initial significance map of the full resolution is obtained.
Step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs;
in a further embodiment, the peak-valley detection is performed by using the histogram statistical features of the depth map, and the steps are as follows:
step 3-1, dividing the depth values of all pixels in the depth map into 256 intervals, and counting the number of the pixels of the depth values in each interval range to obtain a statistical histogram;
step 3-2, calculating a derivative of the histogram statistic value to obtain the growth rate of each position corresponding to the abscissa of the histogram, and forming a vector alpha ═ alpha1,α2,…,α256};
Step 3-3, taking alphaiSymbol λ of 1,2, …, 256αiAnd forming them into a vector lambda in orderα={λα1,λα2,…,λα256},αiSign of (a)αiThe concrete formula of (1) is as follows:
step 3-4, vector λαCarrying out mean value filtering, and executing the operation of the step 3-3 on the filtered result to obtain a new digital string lambdaβ={λβ1,λβ2,…,λβ256};
Step 3-5, vector lambda is aligned by adopting a template matching modeβCarrying out jump detection; there are 4 types of hopping: [1, -1],[1,0,-1]The jump position is the peak position Pp;[-1,1],[-1,0,1]Corresponding to the position P of the wave trought。
Step 4, estimating a Zero Parallax Area (ZPA) of the depth map;
in a further embodiment, the specific step of estimating the zero-disparity region ZPA of the depth image is as follows:
And 4-2, taking the median as a center, wherein the area within the range of the distance between the front and the back of the center and the distance between the front and the back of the center is ZPA of the scene:
in equation (9), H is the depth-of-field (DOF) of the scene, and σ is the scale parameter.
And 5, dividing a background area and a foreground area in the depth image, adjusting the initial saliency map obtained in the step 2 according to the background area and the foreground area, inhibiting the saliency value of the background area, and obtaining an improved saliency map.
In a further embodiment, the improved saliency map is obtained by the following steps:
step 5-1, determining a depth value corresponding to a peak-valley position which is behind the zero parallax zone ZPA and is closest to the zero parallax zone ZPA, namely a final threshold value T of the background estimation:
T=min(p),st p∈{Pp,Ptand p > ZPA (10)
Step 5-2, in the depth image, taking an area with a depth value larger than a background threshold value T as a background part, and taking a part with a depth value smaller than T as a foreground area, and thus determining whether a pixel at a corresponding position in the saliency map belongs to the background part or the foreground part; suppressing the significance value of the background part in the significance map, and reserving the significance value of the foreground part in the significance map to obtain an improved significance map, wherein the suppression formula of the significance value of the background part is as follows:
in the formula, depkIs a pixel point IkCorresponding to the depth value on the depth image, S (I)k) Is the initial saliency value, S' (I) of the background portionk) The significance value of the background part after inhibition is shown.
Step 6, performing superpixel segmentation on the original image by adopting a superpixel segmentation algorithm, and then optimizing the saliency map obtained in the step 5 to obtain a final saliency area;
in a further embodiment, the step of optimizing the saliency map in step 5 based on the super-pixel pairs is as follows:
step 6-1, initializing a clustering center: setting the number of superpixels as C, length in two-dimensional spaceFor the interval, periodically sampling the depth image, taking each sampling point as an initial clustering center, setting the category labels of all the pixels of the initial clustering centers to be 1,2, … and C, setting the category labels of all the pixels of the non-clustering centers to be-1, setting the distance between the pixels of the non-clustering centers and the clustering centers to be infinite, and setting N to be the total number of the pixels in the whole depth image;
step 6-2, for each clustering center IcC, respectively calculating the cluster center and each pixel point I in the 2s × 2s neighborhood search range of the cluster centeri1, 2., a distance of 2s × 2s, the distance calculation formula is as follows:
wherein depcFor clustering central pixel point IcDepth value of uc,νcIs IcAbscissa and ordinate in the image; depiIs a pixel point IiDepth value of ui,νiIs IiThe horizontal and vertical coordinates in the image, m is the compactness adjusting parameter of the super pixel;
each non-clustering center pixel point is searched by a plurality of surrounding clustering center points, the clustering center corresponding to the minimum distance value is taken as the clustering center of the pixel point, and the clustering center is set as a category label same as the clustering center, so that a super-pixel segmentation result is obtained;
6-3, calculating the depth mean value and the horizontal and vertical coordinate mean values of the pixel points in each super pixel, taking the depth mean value and the coordinate mean value of each super pixel as a new clustering center of the super pixel, and repeating the step 6-2 until the clustering center to which each pixel point belongs does not change any more;
step 6-4, counting the number of pixels contained in each super pixel, and merging the number of pixels with the super pixel with the nearest coordinate position in the adjacent super pixels when the number of pixels is smaller than a set minimum value e; after combination, all the super-pixels R are obtainedcC ═ 1,2,. C ', where C' ≦ C;
6-5, according to the super pixel RcOptimizing the significance result obtained in step 5, i.e. if Ik∈RcThen the pixel IkFinal significance value of S ″ (I)k) Comprises the following steps:
wherein, | RcIs at the super-pixel RcThe number of pixels contained in (a).
The present invention is further illustrated by the following specific examples.
Example 1
As shown in fig. 1, a method for detecting a salient region in a depth image includes the following steps:
step 1, for a depth image I, for each pixel I in the imagekSeparately extracting gradient featuresThe original image is shown in fig. 2 (a), and the depth image is shown in fig. 2 (b);
step 1-1, traversing all pixel points of the depth image I to obtain a gradient vector of each pixel point, and counting the pixel points IkN, N is the total number of pixels, and its gradient vector (dr) isk,dck) The calculation formula is as follows:
drk=(dep(r+1,c)-dep(r-1,c)/2 (1)
dck=(dep(r,c+1)-dep(r,c-1)/2 (2)
wherein r and c correspond to rows and columns of image coordinates, dep (r, c) represents the depth value of the r-th row and c-th column in the depth image I;
step 1-2, traversing all pixel points to obtain the gradient characteristic of each point, and obtaining a pixel point IkCharacteristic of gradient ofThe method specifically comprises the following steps:
wherein ε is 0.02 and Maximun is GaAnd GbMaximum value of (a), maximum 600 in this example;
step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation method according to the gradient characteristics in the step 1k) Obtaining an initial saliency map of the same resolution, specifically comprising the following steps:
step 2-1, normalizing two elements of all gradient feature vectors to an interval [0, 255%]Rounding off to obtain integer value and pixel point IkAfter being normalized, the gradient features ofThereby the gradient characteristic values of all pixel pointsAll correspond to [0,255]An integer of up to 256 different values, notedThe same can be obtained
Step 2-2, obtaining characteristic values according to the calculation mode of the global contrastCorresponding significance values:
where n is 256, the total number of feature values extracted from the depth image, and fjRepresentsThe probability of occurrence in the image is,is composed ofAnda distance metric function of the two features; for characteristic valueIn the same way, the corresponding significance values are obtained:
step 2-3, the corresponding significance values of the pixels with the same characteristic value are also the same, and the pixel I is subjected tokIf its characteristic value isThe initial saliency value for that pixel is then:
wherein, waAnd wbAre weight parameters, all set to 0.5; for each pixel, according to the characteristic value of the pixel, the significance value of the pixel can be obtained, and therefore an initial significance map of the full resolution is obtained;
step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs, and comprising the following steps:
step 3-1, dividing the depth values of all pixels in the depth map into 256 intervals, and counting the number of the pixels of the depth values in each interval range to obtain a statistical histogram;
step 3-2, calculating a derivative of the histogram statistic value to obtain the growth rate of each position corresponding to the abscissa of the histogram, and forming a vector alpha ═ alpha1,α2,…,α256};
Step 3-3, taking alpha according to a formula (8)iSign of (a)αiAnd forming them into a vector lambda in orderα={λα1,λα2,…,λα256}:
Step 3-4, vector λαCarrying out mean value filtering, repeating the operation of the step 3-3 once on the filtered result to obtain a new digital string lambdaβ={λβ1,λβ2,…,λβ256};
Step 3-5, vector lambda is matched in a template matching modeβCarrying out jump detection; there are 4 types of hopping: [1, -1],[1,0,-1]The jump position is the peak position Pp;[-1,1],[-1,0,1]Corresponding to the position P of the wave trought。
Step 4, estimating a Zero Parallax Area (ZPA) of the depth image, which comprises the following steps:
And 4-2, determining areas around the median as ZPA of the scene:
in equation (9), H is the depth-of-field (DOF) of the scene, and σ is 0.1, which is a scale parameter.
Step 5, dividing a background area and a foreground area in the depth image, adjusting the saliency map obtained in the step 2 according to the background area, suppressing the saliency value of the background area, and obtaining an improved saliency map, wherein the steps are as follows:
step 5-1, determining a depth value corresponding to a peak-valley position which is behind the zero parallax zone ZPA and is closest to the zero parallax zone ZPA, namely a final threshold value T of the background estimation:
T=min(p),st p∈{Pp,Ptand p > ZPA (10)
Step 5-2, in the depth image, taking an area with a depth value larger than a background threshold value T as a background part, and taking a part with a depth value smaller than T as a foreground area, and thus determining whether a pixel at a corresponding position in the saliency map belongs to the background part or the foreground part; suppressing the significance value of the background part in the significance map, and reserving the significance value of the foreground part in the significance map to obtain an improved significance map, wherein the suppression formula of the significance value of the background part is as follows:
in the formula, depkIs a pixel point IkCorresponding to the depth value on the depth image, S (I)k) Is the initial saliency value, S' (I) of the background portionk) For background partial significance value after suppression
And 6, further improving and obtaining a final significant region detection result by adopting a method based on super-pixel division, wherein the steps are as follows:
step 6-1, initializing a clustering center: setting the number of superpixels as the number of superpixels C1600 and the length in the two-dimensional space in the whole depth imagePeriodically sampling the depth image for intervals, taking each sampling point as an initial clustering center, setting the class labels of all pixels in the initial clustering centers to be 1,2, … and C, setting the class labels of all pixels in non-clustering centers to be-1, setting the distance between the pixels and the clustering centers to be infinite, setting N to be the total number of pixels in the whole depth image, and for a typical depth image with the resolution of 640 multiplied by 480, corresponding interval length to the depth image is equal to
Step 6-2, for each clustering center IcC, calculating the cluster center and each pixel point I in the 28 × 28 neighborhood search range respectivelyi1, 2., a distance of 2s × 2s, the distance calculation formula is as follows:
wherein depcFor clustering central pixel point IcDepth value of uc,νcIs IcAbscissa and ordinate in the image; depiIs a pixel point IiDepth value of ui,νiIs IiAbscissa and ordinate in the image; the compactness adjusting parameter m of the super pixel is 40;
each non-clustering center pixel point is searched by a plurality of surrounding clustering center points, the clustering center corresponding to the minimum distance value is taken as the clustering center of the pixel point, and the clustering center is set as a category label same as the clustering center, so that a super-pixel segmentation result is obtained;
6-3, calculating the depth mean value and the horizontal and vertical coordinate mean values of the pixel points in each super pixel, taking the depth mean value and the coordinate mean value of each super pixel as a new clustering center of the super pixel, and repeating the step 6-2 until the clustering center to which each pixel point belongs does not change any more; in the embodiment, 10 iterations can obtain ideal effects on most pictures, so that 10 iterations are selected;
step 6-4, setting the minimum value e of the number of pixels contained in the superpixel to be 20, and combining a morphological region smaller than e with a neighborhood thereof; after combination, all the super-pixels R are obtainedcC ═ 1,2,. C ', where C' ≦ C;
6-5, according to the super pixel RcOptimizing the significance result obtained in step 5, i.e. if Ik∈RcThen the pixel IkFinal significance value of S ″ (I)k) Comprises the following steps:
wherein, | RcIs at the super-pixel RcThe number of pixels contained in (a). As shown in fig. 2 (c), the closer the area in the graph is to white, the higher the saliency value of the area is, and the closer to black, the lower the saliency value is.
The invention adopts the gradient characteristic of the depth value as the calculation basis of the overall contrast, reduces noise interference, is not influenced by the change of the range of the depth value, and improves the accuracy of the detection result; by using a background area division method, the significance of a background and a highlighted target area is effectively inhibited, and the stability of a detection result is improved; the saliency of the foreground part is optimized by dividing the superpixels, so that a saliency map of a full resolution is obtained through calculation, a reliable saliency region is provided for target detection, target identification, scene understanding and the like, and the acquisition capability of an image region of interest is improved.
Claims (7)
1. A method for detecting a salient region in a depth image is characterized by comprising the following steps:
Step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation mode according to the gradient features extracted in the step 1k) Obtaining an initial saliency map of the same resolution;
step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs;
step 4, estimating a zero parallax area ZPA of the depth image;
step 5, dividing a background area and a foreground area in the depth image according to the peak and trough detection result and the zero parallax area ZPA, adjusting the saliency map obtained in the step 2 according to the background area and the foreground area, and inhibiting the saliency value of the background area to obtain an improved saliency map;
and 6, performing superpixel segmentation on the original image by adopting a superpixel segmentation algorithm, and then optimizing the saliency map obtained in the step 5 to obtain a final saliency area.
2. The method according to claim 1, wherein the extracting gradient features of the depth map in step 1 specifically comprises the following steps:
step 1-1, traversing all pixel points of the depth image I to obtain a gradient vector of each pixel point, and counting the pixel points IkN, N is the total number of pixels, and its gradient vector (dr) isk,dck) The calculation formula is as follows:
drk=(dep(r+1,c)-dep(r-1,c)) /2 (1)
dck=(dep(r,c+1)-dep(r,c-1)) /2 (2)
wherein r and c correspond to rows and columns of image coordinates, dep (r, c) represents the depth value of the r-th row and c-th column in the depth image I;
step 1-2, traversing all pixel points to obtain the gradient characteristic of each point, and obtaining a pixel point IkCharacteristic of gradient ofThe method specifically comprises the following steps:
3. The method for detecting the salient region in the depth image according to claim 1, wherein in step 2, an initial saliency value S (I) of each pixel is calculated by adopting a global contrast method according to gradient featuresk) Obtaining an initial saliency map of the same resolution, specifically comprising the following steps:
step 2-1, normalizing two elements of all gradient feature vectors to an interval [0, 255%]Rounding off to obtain integer value and pixel point IkAfter being normalized, the gradient features ofThereby the gradient characteristic values of all pixel pointsAll correspond to [0,255]An integer of up to 256 different values, notedThe same can be obtained
Step 2-2, obtaining characteristic values according to the calculation mode of the global contrastCorresponding significance values:
where n is 256, the total number of feature values extracted from the depth image, and fjRepresentsThe probability of occurrence in the image is,is composed ofAnda distance metric function of the two features; for characteristic valueIn the same way, the corresponding significance values are obtained:
step 2-3, the corresponding significance values of the pixels with the same characteristic value are also the same, and the pixel I is subjected tokIf its characteristic value isThe initial saliency value for that pixel is then:
wherein, waAnd wbIs a weight parameter; for each pixel, according to the characteristic value of the pixel, the significance value of the pixel can be obtained, and therefore the initial significance map of the full resolution is obtained.
4. The method for detecting the significant region in the depth image according to claim 1, wherein the step 3 performs peak-valley detection by using histogram statistical features of the depth image, and comprises the following steps:
step 3-1, dividing the depth values of all pixels in the depth map into 256 intervals, and counting the number of the pixels of the depth values in each interval range to obtain a statistical histogram;
step 3-2, calculating a derivative of the histogram statistic value to obtain the growth rate of each position corresponding to the abscissa of the histogram, and forming a vector alpha ═ alpha1,α2,…,α256};
Step 3-3, taking alphaiSymbol λ of 1,2, …, 256αiAnd forming them into a vector lambda in orderα={λα1,λα2,…,λα256},αiSign of (a)αiThe concrete formula of (1) is as follows:
step 3-4, vector λαCarrying out mean value filtering, and executing the operation of the step 3-3 on the filtered result to obtain a new digital string lambdaβ={λβ1,λβ2,…,λβ256};
Step 3-5, vector lambda is aligned by adopting a template matching modeβCarrying out jump detection; a total of 4Hopping type: [1, -1],[1,0,-1]The jump position is the peak position Pp;[-1,1],[-1,0,1]Corresponding to the position P of the wave trought。
5. The method according to claim 2, wherein the step 4 of estimating the zero-disparity region ZPA of the depth image comprises the following specific steps:
And 4-2, taking the median as a center, wherein an area within a range of the distance between the front and the back of the center and the distance between the front and the back of the center are zero parallax areas ZPA of the depth image:
in the formula, H is the depth of field of the scene, and σ is a proportional parameter.
6. The method according to claim 4, wherein in step 5, the background area and the foreground area in the depth image are divided, and the saliency map obtained in step 2 is adjusted accordingly to suppress the saliency value of the background area, so as to obtain an improved saliency map, and the steps are as follows:
step 5-1, determining a depth value corresponding to a peak-valley position which is behind the zero parallax zone ZPA and is closest to the zero parallax zone ZPA, namely a final threshold value T of the background estimation:
T=min(p),st p∈{Pp,Ptand p > ZPA (10)
Step 5-2, in the depth image, taking an area with a depth value larger than a background threshold value T as a background part, and taking a part with a depth value smaller than T as a foreground area, and thus determining whether a pixel at a corresponding position in the saliency map belongs to the background part or the foreground part; suppressing the significance value of the background part in the significance map, and reserving the significance value of the foreground part in the significance map to obtain an improved significance map, wherein the suppression formula of the significance value of the background part is as follows:
in the formula, depkIs a pixel point IkCorresponding to the depth value on the depth image, S (I)k) Is the initial saliency value, S' (I) of the background portionk) The significance value of the background part after inhibition is shown.
7. The method for detecting the salient region in the depth image according to claim 1, wherein the original image is subjected to superpixel segmentation by using a superpixel segmentation algorithm in step 6, and then the salient image obtained in step 5 is optimized to obtain a final salient region, and the specific steps are as follows:
step 6-1, initializing a clustering center: setting the number of superpixels as C, length in two-dimensional spaceFor the interval, periodically sampling the depth image, taking each sampling point as an initial clustering center, setting the category labels of all the pixels of the initial clustering centers to be 1,2, … and C, setting the category labels of all the pixels of the non-clustering centers to be-1, setting the distance between the pixels of the non-clustering centers and the clustering centers to be infinite, and setting N to be the total number of the pixels in the whole depth image;
step 6-2, for each clustering center IcC, respectively calculating the cluster center and each pixel point I in the 2s × 2s neighborhood search range of the cluster centeri1, 2., a distance of 2s × 2s, the distance calculation formula is as follows:
wherein depcFor clustering central pixel point IcDepth value of uc,νcIs IcAbscissa and ordinate in the image; depiIs a pixel point IiDepth value of ui,νiIs IiAbscissa and ordinate in the image;
each non-clustering center pixel point is searched by a plurality of surrounding clustering center points, the clustering center corresponding to the minimum distance value is taken as the clustering center of the pixel point, and the clustering center is set as a category label same as the clustering center, so that a super-pixel segmentation result is obtained;
6-3, calculating the depth mean value and the horizontal and vertical coordinate mean values of the pixel points in each super pixel, taking the depth mean value and the coordinate mean value of each super pixel as a new clustering center of the super pixel, and repeating the step 6-2 until the clustering center to which each pixel point belongs does not change any more;
step 6-4, counting the number of pixels contained in each super pixel, and merging the number of pixels with the super pixel with the nearest coordinate position in the adjacent super pixels when the number of pixels is smaller than a set minimum value e; after combination, all the super-pixels R are obtainedcC ═ 1,2,. C ', where C' ≦ C;
6-5, according to the super pixel RcOptimizing the significance result obtained in step 5, i.e. if Ik∈RcThen the pixel IkFinal significance value of S ″ (I)k) Comprises the following steps:
wherein, | RcIs at the super-pixel RcThe number of pixels contained in (a).
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810757983.1A CN109118493B (en) | 2018-07-11 | 2018-07-11 | Method for detecting salient region in depth image |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810757983.1A CN109118493B (en) | 2018-07-11 | 2018-07-11 | Method for detecting salient region in depth image |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109118493A CN109118493A (en) | 2019-01-01 |
CN109118493B true CN109118493B (en) | 2021-09-10 |
Family
ID=64862700
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810757983.1A Active CN109118493B (en) | 2018-07-11 | 2018-07-11 | Method for detecting salient region in depth image |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109118493B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111986245A (en) * | 2019-05-23 | 2020-11-24 | 北京猎户星空科技有限公司 | Depth information evaluation method and device, electronic equipment and storage medium |
CN114640850B (en) * | 2022-02-28 | 2024-06-18 | 上海顺久电子科技有限公司 | Video image motion estimation method, display device and chip |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103208123A (en) * | 2013-04-19 | 2013-07-17 | 广东图图搜网络科技有限公司 | Image segmentation method and system |
CN104574375A (en) * | 2014-12-23 | 2015-04-29 | 浙江大学 | Image significance detection method combining color and depth information |
CN105550678A (en) * | 2016-02-03 | 2016-05-04 | 武汉大学 | Human body motion feature extraction method based on global remarkable edge area |
CN107169487A (en) * | 2017-04-19 | 2017-09-15 | 西安电子科技大学 | The conspicuousness object detection method positioned based on super-pixel segmentation and depth characteristic |
CN107240096A (en) * | 2017-06-01 | 2017-10-10 | 陕西学前师范学院 | A kind of infrared and visual image fusion quality evaluating method |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10095953B2 (en) * | 2009-11-11 | 2018-10-09 | Disney Enterprises, Inc. | Depth modification for display applications |
-
2018
- 2018-07-11 CN CN201810757983.1A patent/CN109118493B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103208123A (en) * | 2013-04-19 | 2013-07-17 | 广东图图搜网络科技有限公司 | Image segmentation method and system |
CN104574375A (en) * | 2014-12-23 | 2015-04-29 | 浙江大学 | Image significance detection method combining color and depth information |
CN105550678A (en) * | 2016-02-03 | 2016-05-04 | 武汉大学 | Human body motion feature extraction method based on global remarkable edge area |
CN107169487A (en) * | 2017-04-19 | 2017-09-15 | 西安电子科技大学 | The conspicuousness object detection method positioned based on super-pixel segmentation and depth characteristic |
CN107240096A (en) * | 2017-06-01 | 2017-10-10 | 陕西学前师范学院 | A kind of infrared and visual image fusion quality evaluating method |
Non-Patent Citations (1)
Title |
---|
基于空-频域混合分析的RGB-D数据视觉显著性检测方法;岳娟等;《机器人》;20170930;第39卷(第5期);第652-660页 * |
Also Published As
Publication number | Publication date |
---|---|
CN109118493A (en) | 2019-01-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Lian et al. | Density map regression guided detection network for rgb-d crowd counting and localization | |
Chen et al. | Efficient hierarchical method for background subtraction | |
CN104200485B (en) | Video-monitoring-oriented human body tracking method | |
CN104598883B (en) | Target knows method for distinguishing again in a kind of multiple-camera monitoring network | |
CN109099929B (en) | Intelligent vehicle positioning device and method based on scene fingerprints | |
Gui et al. | A new method for soybean leaf disease detection based on modified salient regions | |
CN104715251B (en) | A kind of well-marked target detection method based on histogram linear fit | |
Fradi et al. | Low level crowd analysis using frame-wise normalized feature for people counting | |
CN112488057A (en) | Single-camera multi-target tracking method utilizing human head point positioning and joint point information | |
CN103735269A (en) | Height measurement method based on video multi-target tracking | |
CN105809673B (en) | Video foreground dividing method based on SURF algorithm and the maximum similar area of merging | |
CN109118493B (en) | Method for detecting salient region in depth image | |
Déniz et al. | Fast and accurate global motion compensation | |
Nguyen et al. | Salient object detection via augmented hypotheses | |
CN111028263B (en) | Moving object segmentation method and system based on optical flow color clustering | |
CN105590086A (en) | Article antitheft detection method based on visual tag identification | |
CN102592277A (en) | Curve automatic matching method based on gray subset division | |
CN117078726A (en) | Different spectrum image registration method based on edge extraction | |
CN108241837B (en) | Method and device for detecting remnants | |
Liu et al. | [Retracted] Mean Shift Fusion Color Histogram Algorithm for Nonrigid Complex Target Tracking in Sports Video | |
CN114511803A (en) | Target occlusion detection method for visual tracking task | |
JP2020086879A (en) | Coordinate transformation matrix estimation method and computer program | |
Lu et al. | Edge and color contexts based object representation and tracking | |
Yu et al. | Motion detection in moving background using a novel algorithm based on image features guiding self-adaptive Sequential Similarity Detection Algorithm | |
Taalimi et al. | Robust multi-object tracking using confident detections and safe tracklets |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |