CN106898005A - A kind of method for realizing interactive image segmentation, device and terminal - Google Patents
A kind of method for realizing interactive image segmentation, device and terminal Download PDFInfo
- Publication number
- CN106898005A CN106898005A CN201710005330.3A CN201710005330A CN106898005A CN 106898005 A CN106898005 A CN 106898005A CN 201710005330 A CN201710005330 A CN 201710005330A CN 106898005 A CN106898005 A CN 106898005A
- Authority
- CN
- China
- Prior art keywords
- pixel
- point
- mask artwork
- mark zone
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
Landscapes
- Image Analysis (AREA)
Abstract
Disclosed herein is a kind of method for realizing interactive image segmentation, device and terminal.Methods described includes:Construction is comprising the geometric figure for smearing track and extension forms mark zone, it is determined that the area-of-interest comprising the mark zone;Generate the input mask artwork of image segmentation algorithm:Using all of pixel in mark zone as the foreground point in mask artwork, using the pixel outside mark zone in area-of-interest as the background dot in mask artwork;The partitioning parameters of each pixel are determined according to depth map and mask artwork, build non-directed graph and the partitioning parameters of each pixel in mask artwork are mapped in non-directed graph, minimal cut maximum-flow algorithm is run according to non-directed graph, the mask artwork after fine segmentation is obtained, the corresponding image in foreground point in the mask artwork after the fine segmentation is partitioned into from original image.The run time of algorithm can be shortened herein, the depth information using image improves the effect of image segmentation.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to a kind of method for realizing interactive image segmentation, device
And terminal.
Background technology
Image segmentation, if refer to by plane picture according to the features such as color, texture, shape be divided into mutually it is disjunct
Dry region, this is a basic technology for practicality of image processing field.Existing image Segmentation Technology has dividing based on threshold value
Segmentation method, the dividing method based on edge, the dividing method based on region, the dividing method based on energy functional, based on graph theory
Dividing method etc..Comparing wherein in Graph-theoretical Approach well-known has GraphCut algorithms and its modified version GrabCut algorithms.
GraphCut algorithms and its modified version GrabCut algorithms, are the interactive image segmentation sides based on area marking
Method.GraphCut algorithms are based on Markov random field (Markov Random Field, MRF) energy minimization framework
A kind of algorithm, advantage is that can carry out global optimum's solution with reference to various knowwhies.GrabCut algorithms are to GraphCut
The improvement of algorithm, GrabCut algorithms by mark out on the original image foreground point (point on the destination object to be extracted) and
Background dot generates mask artwork, and gauss hybrid models (Gaussian is set up to prospect, background color space using artwork and mask artwork
Mixture Model, GMM), energy minimization is completed using the iterative algorithm that can be evolved in GMM parameter learnings, estimation procedure,
The foreground point in image and background dot are ruled out, the target image being made up of foreground point pixel is extracted from artwork.
When carrying out image segmentation using GrabCut algorithms on mobile phone, in order to reduce interactive complexity, generally to user
How to mark and be not strict with, therefore, in the case where the foreground point of user's mark is less, iterations may be a lot, algorithm
Run time is more long, have impact on the experience of user.On the other hand, the GrabCut algorithms in correlation technique are based on coloured image
Image segmentation is carried out, when the color characteristic of the destination object to be extracted is not obvious, using dividing that cromogram is split
Cut effect unsatisfactory.
The content of the invention
The technical problems to be solved by the invention are to provide a kind of method for realizing interactive image segmentation, device and end
End, can shorten the run time of algorithm, and the depth information using image improves the effect of image segmentation.
A kind of method for realizing interactive image segmentation is the embodiment of the invention provides, including:
After detecting the smearing track on original image, construction includes the geometric figure of the smearing track and to described several
What figure is extended to form mark zone, it is determined that one piece of region comprising the smearing track is used as area-of-interest, the sense
Interest region includes the mark zone;Generate the input mask artwork of image segmentation algorithm:By all of pixel in the mark zone
As the foreground point in the mask artwork, using the pixel outside mark zone in the area-of-interest as the back of the body in the mask artwork
Sight spot;
The depth map comprising destination object depth information is obtained, the mask artwork is determined according to the depth map and mask artwork
The partitioning parameters of upper each pixel, the partitioning parameters are used to represent that pixel judgement is the probability and institute of foreground point or background dot
State the depth value difference of pixel and adjacent pixel;
Non-directed graph is built, the partitioning parameters of each pixel in the mask artwork is mapped in the non-directed graph, root
The non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtains the mask artwork after fine segmentation;
Before being partitioned into the mask artwork after the fine segmentation from the original image that the destination object is obtained is shot
The corresponding image in sight spot.
Alternatively, the partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, including:
Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps;Iteration runs E steps and M steps
Suddenly, the iterative process is stopped after iteration operation reaches the condition of convergence;Last time is performed described in the acquisition of M steps
The classification of pixel is defined as the classification of the pixel, and the pixel that last time performs the acquisition of M steps is belonged into certain cluster
Most probable value PmaxIt is defined as the area item partitioning parameters of the pixel, the area item partitioning parameters are that the pixel is sentenced
It is certainly foreground point or the probability of background dot;
Wherein, the E steps and M steps include following treatment respectively:
E steps:The position relationship between depth value and pixel according to each pixel on the mask artwork is by clustering
Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined;Wherein, the classification of pixel includes prospect
Point or background dot;The classification of cluster includes foreground point cluster or background dot cluster;
M steps:GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one
Pixel, the most probable value P according to the pixelmaxCorresponding cluster determines the classification of the pixel.
Alternatively, the partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, is also wrapped
Include:Depth value difference according to the pixel and adjacent pixel determines the border item partitioning parameters of the pixel;
Wherein, to any one pixel, by the depth value absolute difference between the pixel and each neighbouring pixel
Added up, then to adding up and being normalized, cumulative and as the pixel the border item point after being normalized
Cut parameter.
Alternatively, non-directed graph is built, the partitioning parameters of each pixel in the mask artwork is mapped to described undirected
In figure, including:
A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph0And Q1, the hanging point Q0For
Virtual foreground point, the hanging point Q1It is virtual background point;Each is set up on the mask artwork in the plane of the non-directed graph
The mapping point of pixel, in mapping point and the hanging point Q of foreground point0Between set up line, mapping point and institute in background dot
State hanging point Q1Between set up line;
To any one pixel P in mask artworki, by the pixel PiArea item partitioning parameters as the non-directed graph
Middle mapping point Pi' weights, by the pixel PiBorder item partitioning parameters as mapping point P in the non-directed graphi' with it is hanging
Point Q0Or Q1Between line weights.
Alternatively, it is described the non-directed graph is processed according to minimal cut-maximum-flow algorithm, after obtaining fine segmentation
Mask artwork, including:
Iteration performs following step C and D steps, and iterative process is stopped after iteration operation reaches the condition of convergence, will be described
Each pixel in prospect point set Q is used as the foreground point in the mask artwork after fine segmentation;
Wherein, step C and D steps include following treatment respectively:
Step C:One part of pixel in non-directed graph is divided into and vacantly point Q0Similar foreground point, by being divided into prospect
The pixel of point constitutes prospect point set Q;
D steps:The weights summation of the prospect point set Q is calculated, the weights summation is institute in the prospect point set Q
Have foreground point weights and, along with all foreground points in the prospect point set Q and hanging point Q0Between line weights and;
Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability.
Alternatively, the original image that the shooting destination object is obtained is cromogram, artwork master or infrared figure.
Alternatively, construct the geometric figure comprising the smearing track and the geometric figure is extended to form mark
Area, it is determined that one piece of region comprising the smearing track is used as area-of-interest, including:
The minimum enclosed rectangle of the smearing track is constructed, the length L according to the smearing track is external to the minimum
Rectangle is extended to form mark zone;
For geometry template is applied mechanically in the smearing track, the smearing rail is included according to the geometry template generation
The area-of-interest of mark, if the area-of-interest does not include the mark zone, the area-of-interest is extended to
Comprising the mark zone;Or expansion process formation area-of-interest is carried out to the mark zone.
Alternatively, the minimum enclosed rectangle of the construction smearing track, the length L according to the smearing track is to institute
Minimum enclosed rectangle is stated to be extended to form mark zone, including:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum horizontal stroke
Coordinate value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, four coordinates on summit of the minimum enclosed rectangle are successively
It is:(x_min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark
Four apex coordinates in area are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a),
(x_max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants.
Alternatively, before the input mask artwork of generation image segmentation algorithm, also include:
If currently detected smearing track is directed to the first time mark of the original image, will be currently detected
The mark zone determined of smearing track as final mark zone, by currently detected smearing track determine it is interested
Region is used as final area-of-interest;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
Whether have between the mark zone that the smearing track that the mark zone and last time that the smearing track for measuring is determined are detected is determined
Occur simultaneously, if occuring simultaneously, then the smearing rail that the mark zone determined currently detected smearing track is detected with the last time
The mark zone that mark is determined is merged, using the mark zone after merging as final mark zone;If do not occured simultaneously, ought
Before the mark zone determined of the smearing track that detects as final mark zone;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
The area-of-interest that area-of-interest that the smearing track for measuring is determined is determined with the last smearing track for detecting it
Between whether have common factor, if occuring simultaneously, then the area-of-interest determined currently detected smearings track and last inspection
The area-of-interest that the smearing track for measuring is determined is merged, using the area-of-interest after merging as final interested
Region;If do not occured simultaneously, the area-of-interest that currently detected smearing track is determined is used as final interested
Region.
Alternatively, described image partitioning algorithm is GrabCut algorithms.
The embodiment of the present invention additionally provides a kind of device for realizing interactive image segmentation, including:
Pretreatment module, after detecting the smearing track on original image, construction is several comprising the smearing track
What figure simultaneously is extended to form mark zone to the geometric figure, it is determined that one piece of region comprising the smearing track is used as sense
Interest region, the area-of-interest includes the mark zone;Generate the input mask artwork of image segmentation algorithm:By the mark
All of pixel is used as the foreground point in the mask artwork in area, using the pixel outside mark zone in the area-of-interest as institute
State the background dot in mask artwork;
Partitioning parameters computing module, for obtaining the depth map comprising destination object depth information, according to the depth map
With the partitioning parameters that mask artwork determines each pixel on the mask artwork, it is prospect that the partitioning parameters are used to represent that pixel is adjudicated
The probability and the pixel of point or background dot and the depth value difference of adjacent pixel;
Mask artwork adjusting module, for building non-directed graph, the partitioning parameters of each pixel in the mask artwork is reflected
It is mapped in the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtains covering after fine segmentation
Mould figure;
Output module, after being partitioned into the fine segmentation from the original image that the destination object is obtained is shot
Mask artwork in the corresponding image in foreground point.
Alternatively, partitioning parameters computing module, for determining institute according to the depth map and mask artwork in the following ways
State the partitioning parameters of each pixel on mask artwork:Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E
Step and M steps;Iteration runs E steps and M steps, and the iteration mistake is stopped after iteration operation reaches the condition of convergence
Journey;The classification that last time performs the pixel that M steps are obtained is defined as the classification of the pixel, last time is performed
The pixel that M steps are obtained belongs to the most probable value P of certain clustermaxIt is defined as the area item partitioning parameters of the pixel,
The area item partitioning parameters are that the pixel judgement is foreground point or the probability of background dot;
Wherein, the E steps and M steps include following treatment respectively:
E steps:The position relationship between depth value and pixel according to each pixel on the mask artwork is by clustering
Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined;Wherein, the classification of pixel includes prospect
Point or background dot;The classification of cluster includes foreground point cluster or background dot cluster;
M steps:GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one
Pixel, the most probable value P according to the pixelmaxCorresponding cluster determines the classification of the pixel.
Alternatively, partitioning parameters computing module, is additionally operable to be determined according to the depth map and mask artwork in the following ways
The partitioning parameters of each pixel on the mask artwork:Depth value difference according to the pixel and adjacent pixel determines the pixel
Border item partitioning parameters;
Wherein, to any one pixel, by the depth value absolute difference between the pixel and each neighbouring pixel
Added up, then to adding up and being normalized, cumulative and as the pixel the border item point after being normalized
Cut parameter.
Alternatively, mask artwork adjusting module, for building non-directed graph and will be every in the mask artwork in the following ways
The partitioning parameters of one pixel are mapped in the non-directed graph:
A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph0And Q1, the hanging point Q0For
Virtual foreground point, the hanging point Q1It is virtual background point;Each is set up on the mask artwork in the plane of the non-directed graph
The mapping point of pixel, in mapping point and the hanging point Q of foreground point0Between set up line, mapping point and institute in background dot
State hanging point Q1Between set up line;
To any one pixel P in mask artworki, by the pixel PiArea item partitioning parameters as the non-directed graph
Middle mapping point Pi' weights, by the pixel PiBorder item partitioning parameters as mapping point P in the non-directed graphi' with it is hanging
Point Q0Or Q1Between line weights.
Alternatively, mask artwork adjusting module, in the following ways according to minimal cut-maximum-flow algorithm to described undirected
Figure is processed, and obtains the mask artwork after fine segmentation:
Iteration performs following step C and D steps, and iterative process is stopped after iteration operation reaches the condition of convergence, will be described
Each pixel in prospect point set Q is used as the foreground point in the mask artwork after fine segmentation;
Wherein, step C and D steps include following treatment respectively:
Step C:One part of pixel in non-directed graph is divided into and vacantly point Q0Similar foreground point, by being divided into prospect
The pixel of point constitutes prospect point set Q;
D steps:The weights summation of the prospect point set Q is calculated, the weights summation is institute in the prospect point set Q
Have foreground point weights and, along with all foreground points in the prospect point set Q and hanging point Q0Between line weights and;
Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability.
Alternatively, the original image that the shooting destination object is obtained is cromogram, artwork master or infrared figure.
Alternatively, pretreatment module, for constructing the geometric figure comprising the smearing track and right in the following ways
The geometric figure is extended to form mark zone, it is determined that one piece of region comprising the smearing track is used as area-of-interest:
The minimum enclosed rectangle of the smearing track is constructed, the length L according to the smearing track is external to the minimum
Rectangle is extended to form mark zone;
For geometry template is applied mechanically in the smearing track, the smearing rail is included according to the geometry template generation
The area-of-interest of mark, if the area-of-interest does not include the mark zone, the area-of-interest is extended to
Comprising the mark zone;Or expansion process formation area-of-interest is carried out to the mark zone.
Alternatively, pretreatment module, minimum enclosed rectangle and root for constructing the smearing track in the following ways
The minimum enclosed rectangle is extended to form mark zone according to the length L of the smearing track:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum horizontal stroke
Coordinate value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, four coordinates on summit of the minimum enclosed rectangle are successively
It is:(x_min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark
Four apex coordinates in area are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a),
(x_max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants.
Alternatively, pretreatment module, was additionally operable to before the input mask artwork of generation image segmentation algorithm, was marked area
With the adjustment of area-of-interest:
If currently detected smearing track is directed to the first time mark of the original image, will be currently detected
The mark zone determined of smearing track as final mark zone, by currently detected smearing track determine it is interested
Region is used as final area-of-interest;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
Whether have between the mark zone that the smearing track that the mark zone and last time that the smearing track for measuring is determined are detected is determined
Occur simultaneously, if occuring simultaneously, then the smearing rail that the mark zone determined currently detected smearing track is detected with the last time
The mark zone that mark is determined is merged, using the mark zone after merging as final mark zone;If do not occured simultaneously, ought
Before the mark zone determined of the smearing track that detects as final mark zone;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
The area-of-interest that area-of-interest that the smearing track for measuring is determined is determined with the last smearing track for detecting it
Between whether have common factor, if occuring simultaneously, then the area-of-interest determined currently detected smearings track and last inspection
The area-of-interest that the smearing track for measuring is determined is merged, using the area-of-interest after merging as final interested
Region;If do not occured simultaneously, the area-of-interest that currently detected smearing track is determined is used as final interested
Region.
Alternatively, described image partitioning algorithm is GrabCut algorithms.
The embodiment of the present invention additionally provides a kind of terminal, including the above-mentioned device for realizing interactive image segmentation.
Set forth herein a kind of method for realizing interactive image segmentation, device and terminal, construction is comprising smearing track
Geometric figure and extension form mark zone, it is determined that the area-of-interest comprising the mark zone;Generate the defeated of image segmentation algorithm
Enter mask artwork:Using all of pixel in mark zone as the foreground point in mask artwork, by the picture outside mark zone in area-of-interest
Element is used as the background dot in mask artwork;The partitioning parameters of each pixel, the partitioning parameters are determined according to depth map and mask artwork
For representing that pixel judgement is the probability and the pixel of foreground point or background dot and the depth value difference of adjacent pixel, build
Simultaneously be mapped to the partitioning parameters of each pixel in the mask artwork in the non-directed graph by non-directed graph, according to the non-directed graph
Operation minimal cut MinCut- max-flow MaxFlow algorithms, obtain the mask artwork after fine segmentation, from the shooting destination object
The corresponding image in foreground point in the mask artwork after the fine segmentation is partitioned into the original image for being obtained.The technical side of this paper
The prospect that case can expand image segmentation algorithm mark by image preprocessing is counted out, and reduces the back of the body of image segmentation algorithm mark
Sight spot number, so that shorten the run time of image segmentation algorithm, based on depth map operation image partitioning algorithm, so as to optimize figure
As the effect of segmentation.
Brief description of the drawings
Fig. 1 is the hardware architecture diagram for realizing each optional mobile terminal of embodiment one of the invention;
Fig. 2 is the wireless communication system schematic diagram of mobile terminal as shown in Figure 1;
Fig. 3 is a kind of method flow diagram for realizing interactive image segmentation of the embodiment of the present invention 1;
Fig. 4 is a kind of schematic device for realizing interactive image segmentation of the embodiment of the present invention 2;
Fig. 5-a are the schematic diagram of original image and user's smearing track in present invention application example 1;
Fig. 5-b be the present invention application example 1 in by smearing Track Pick-up mark zone and the schematic diagram of area-of-interest;
Fig. 5-c are the schematic diagram that the present invention applies the mask artwork generated by mark zone and area-of-interest in example 1;
Fig. 5-d are the schematic diagram of the depth map of present invention application example 1;
Fig. 5-e are the schematic diagram of the non-directed graph of present invention application example 1;
Fig. 5-f are the schematic diagram of the mask artwork after fine segmentation in present invention application example 1;
Fig. 5-g are the schematic diagram that the present invention applies the destination object being partitioned into example 1;
Fig. 6-a-1 are the schematic diagram of original image and user's first time smearing track in present invention application example 2;
Fig. 6-a-2 are second schematic diagram of smearing track of original image and user in present invention application example 2;
Fig. 6-b-1 be the present invention application example 2 in by first time smear Track Pick-up mark zone and area-of-interest
Schematic diagram;
Fig. 6-b-2 are by second mark zone and area-of-interest for smearing Track Pick-up in present invention application example 2
Schematic diagram;
Fig. 6-c-1 be the present invention application example 2 in smear for the first time generation mask artwork schematic diagram;
Fig. 6-c-2 are second schematic diagram of the mask artwork of smearing generation in present invention application example 2.
The realization of the object of the invention, functional characteristics and advantage will be described further referring to the drawings in conjunction with the embodiments.
Specific embodiment
Technical scheme is described in detail below in conjunction with drawings and Examples.
The mobile terminal of each embodiment of the application is realized referring now to Description of Drawings.In follow-up description, use
For represent element such as " module ", " part " or " unit " suffix only for being conducive to explanation of the invention, itself
Not specific meaning.Therefore, " module " can be used mixedly with " part ".
Mobile terminal can be implemented in a variety of manners.For example, the terminal described in the present invention can include such as moving
Phone, smart phone, notebook computer, digit broadcasting receiver, PDA (personal digital assistant), PAD (panel computer), PMP
The mobile terminal of (portable media player), guider etc. and such as numeral TV, desktop computer etc. are consolidated
Determine terminal.Hereinafter it is assumed that terminal is mobile terminal.However, it will be understood by those skilled in the art that, except being used in particular for movement
Outside the element of purpose, construction according to the embodiment of the present invention can also apply to the terminal of fixed type.
Fig. 1 is to realize the application one hardware architecture diagram of optional mobile terminal of each embodiment.
Mobile terminal 1 00 can include wireless communication unit 110, A/V (audio/video) input block 120, user input
Unit 130, sensing unit 140, output unit 150, memory 160, interface unit 170, controller 180 and power subsystem 190
Etc..
Fig. 1 shows the mobile terminal 1 00 with various assemblies, it should be understood that being not required for implementing all showing
The component for going out.More or less component can alternatively be implemented.The element of mobile terminal 1 00 will be discussed in more detail below.
Wireless communication unit 110 can generally include one or more assemblies, and it allows mobile terminal 1 00 and radio communication
Radio communication between system or network.For example, wireless communication unit 110 can include that broadcasting reception module 111, movement are logical
At least one of letter module 112, wireless Internet module 113, short range communication module 114 and location information module 115.
Broadcasting reception module 111 receives broadcast singal and/or broadcast via broadcast channel from external broadcast management server
Relevant information.Broadcast channel can include satellite channel and/or terrestrial channel.Broadcast management server can be generated and sent
The broadcast singal and/or broadcast related information generated before the server or reception of broadcast singal and/or broadcast related information
And send it to the server of terminal.Broadcast singal can include TV broadcast singals, radio signals, data broadcasting
Signal etc..And, broadcast singal may further include the broadcast singal combined with TV or radio signals.Broadcast phase
Pass information can also be provided via mobile communications network, and in said case, broadcast related information can be by mobile communication
Module 112 is received.Broadcast singal can exist in a variety of manners, for example, it can be with the electricity of DMB (DMB)
The form of sub- program guide (EPG), the electronic service guidebooks (ESG) of digital video broadcast-handheld (DVB-H) etc. and exist.Extensively
Broadcasting receiver module 111 can receive signal broadcast by using various types of broadcast systems.Especially, broadcasting reception module
111 can be by using such as multimedia broadcasting-ground (DMB-T), DMB-satellite (DMB-S), digital video
Broadcasting-Handheld (DVB-H), Radio Data System, the received terrestrial digital broadcasting integrated service of forward link media (MediaFLO@)
Etc. (ISDB-T) digit broadcasting system receives digital broadcasting.Broadcasting reception module 111 may be constructed such that and be adapted to provide for extensively
Broadcast the various broadcast systems and above-mentioned digit broadcasting system of signal.Via broadcasting reception module 111 receive broadcast singal and/
Or broadcast related information can be stored in memory 160 (or other types of storage medium).
Mobile communication module 112 sends radio signals to base station (for example, access point, node B etc.), exterior terminal
And at least one of server and/or receive from it radio signal.Such radio signal can be logical including voice
Words signal, video calling signal or the various types of data for sending and/or receiving according to text and/or Multimedia Message.
Wireless Internet module 113 supports the Wi-Fi (Wireless Internet Access) of mobile terminal.The module can be with internal or external
Be couple to terminal.Wi-Fi (Wireless Internet Access) technology involved by the module can include WLAN (WLAN) (Wi-Fi),
Wibro (WiMAX), Wimax (worldwide interoperability for microwave accesses), HSDPA (high-speed downlink packet access) etc..
Short range communication module 114 is the module for supporting junction service.Some examples of short-range communication technology include indigo plant
Tooth TM, radio frequency identification (RFID), Infrared Data Association (IrDA), ultra wide band (UWB), purple honeybee TM etc..
Location information module 115 is the module for checking or obtaining the positional information of mobile terminal.Location information module
115 typical case is GPS (global positioning system).According to current technology, GPS calculate from three or more satellites away from
Information application triangulation from information and correct time information and for calculating, so as to according to longitude, latitude and height
Degree calculates three-dimensional current location information exactly.Currently, three satellites are used simultaneously for calculating the method for position and temporal information
And the error of the position and temporal information for calculating is corrected by using an other satellite.Additionally, GPS can be by real-time
Ground Continuous plus current location information carrys out calculating speed information.
A/V input blocks 120 are used to receive audio or video signal.A/V input blocks 120 can include the He of camera 121
Microphone 122, the static images that 121 pairs, camera is obtained in Video Capture pattern or image capture mode by image capture apparatus
Or the view data of video is processed.Picture frame after treatment may be displayed on display unit 151.Processed through camera 121
Picture frame afterwards can be stored in memory 160 (or other storage mediums) or sent out via wireless communication unit 110
Send, two or more cameras 121 can be provided according to the construction of mobile terminal 1 00.Microphone 122 can be in telephone relation mould
In formula, logging mode, speech recognition mode etc. operational mode sound (voice data), and energy are received via microphone 122
Enough is voice data by such acoustic processing.Audio (voice) data after treatment can be in the case of telephone calling model
The form that being converted to can be sent to mobile communication base station via mobile communication module 112 is exported.Microphone 122 can be implemented various
The noise of type eliminates (or suppression) algorithm and is being received and making an uproar of producing during sending audio signal with eliminating (or suppression)
Sound or interference.
User input unit 130 can generate key input data to control mobile terminal 1 00 according to the order of user input
Various operations.User input unit 130 allow the various types of information of user input, and can include keyboard, metal dome,
Touch pad (for example, detection due to being touched caused by resistance, pressure, electric capacity etc. change sensitive component), roller, shake
Bar etc..Especially, when touch pad is superimposed upon on display unit 151 in the form of layer, touch-screen can be formed.
Sensing unit 140 detects the current state of mobile terminal 1 00, (for example, mobile terminal 1 00 opens or closes shape
State), the presence or absence of the contact (that is, touch input) of the position of mobile terminal 1 00, user for mobile terminal 1 00, mobile terminal
The acceleration or deceleration movement of 100 orientation, mobile terminal 1 00 and direction etc., and generate for controlling mobile terminal 1 00
The order of operation or signal.For example, when mobile terminal 1 00 is embodied as sliding-type mobile phone, sensing unit 140 can be sensed
The sliding-type phone is opened or closed.In addition, sensing unit 140 can detect whether power subsystem 190 provides electric power
Or whether interface unit 170 couples with external device (ED).Sensing unit 140 can include proximity transducer 141.
Interface unit 170 is connected the interface that can pass through with mobile terminal 1 00 as at least one external device (ED).For example,
External device (ED) can include wired or wireless head-band earphone port, external power source (or battery charger) port, wired or nothing
Line FPDP, memory card port, the port for connecting the device with identification module, audio input/output (I/O) end
Mouth, video i/o port, ear port etc..Identification module can be that storage uses each of mobile terminal 1 00 for verifying user
Kind of information and subscriber identification module (UIM), client identification module (SIM), Universal Subscriber identification module (USIM) can be included
Etc..In addition, the device (hereinafter referred to as " identifying device ") with identification module can take the form of smart card, therefore, know
Other device can be connected via port or other attachment means with mobile terminal 1 00.Interface unit 170 can be used for reception and come from
The input (for example, data message, electric power etc.) of the external device (ED) and input that will be received is transferred in mobile terminal 1 00
One or more elements can be used for transmitting data between mobile terminal 1 00 and external device (ED).
In addition, when mobile terminal 1 00 is connected with external base, interface unit 170 can serve as allowing by it by electricity
Power provides to the path of mobile terminal 1 00 from base or can serve as allowing the various command signals being input into from base to pass through it
It is transferred to the path of mobile terminal 1 00.Can serve as recognizing mobile terminal 1 00 from the various command signals or electric power of base input
Whether signal base on is accurately fitted within.Output unit 150 is configured to be provided with vision, audio and/or tactile manner
Output signal (for example, audio signal, vision signal, alarm signal, vibration signal etc.).Output unit 150 can include aobvious
Show unit 151, dio Output Modules 152, alarm unit 153 etc..
Display unit 151 may be displayed on the information processed in mobile terminal 1 00.For example, when mobile terminal 1 00 is in electricity
During words call mode, display unit 151 can show and converse or other communicate (for example, text messaging, multimedia file
Download etc.) related user interface (UI) or graphic user interface (GUI).When mobile terminal 1 00 is in video calling pattern
Or during image capture mode, display unit 151 can show the image of capture and/or the image of reception, show video or figure
UI or GUI of picture and correlation function etc..
Meanwhile, when display unit 151 and touch pad in the form of layer it is superposed on one another to form touch-screen when, display unit
151 can serve as input unit and output device.Display unit 151 can include liquid crystal display (LCD), thin film transistor (TFT)
In LCD (TFT-LCD), Organic Light Emitting Diode (OLED) display, flexible display, three-dimensional (3D) display etc. at least
It is a kind of.Some in these displays may be constructed such that transparence to allow user to be watched from outside, and this is properly termed as transparent
Display, typical transparent display can be, for example, TOLED (transparent organic light emitting diode) display etc..According to specific
Desired implementation method, mobile terminal 1 00 can include two or more display units (or other display devices), for example, moving
Dynamic terminal 100 can include outernal display unit (not shown) and inner display unit (not shown).Touch-screen can be used to detect
Touch input pressure and touch input position and touch input area.
Dio Output Modules 152 can be in call signal reception pattern, call mode, record mould in mobile terminal 1 00
It is that wireless communication unit 110 is received or in memory when under the isotypes such as formula, speech recognition mode, broadcast reception mode
In 160 store voice data transducing audio signal and be output as sound.And, dio Output Modules 152 can provide with
The audio output of the specific function correlation that mobile terminal 1 00 is performed is (for example, call signal receives sound, message sink sound etc.
Deng).Dio Output Modules 152 can include loudspeaker, buzzer etc..
Alarm unit 153 can provide output and be notified to mobile terminal 1 00 with by event.Typical event can be with
Including calling reception, message sink, key signals input, touch input etc..In addition to audio or video is exported, alarm unit
153 can in a different manner provide output with the generation of notification event.For example, alarm unit 153 can be in the form of vibrating
Output is provided, when calling, message or some other entrance communication (incoming communication) are received, alarm list
Unit 153 can provide tactile output (that is, vibrating) to notify to user.Exported by providing such tactile, even if
When in pocket of the mobile phone of user in user, user also can recognize that the generation of various events.Alarm unit 153
The output of the generation of notification event can be provided via display unit 151 or dio Output Modules 152.
Memory 160 can store software program for the treatment and control operation performed by controller 180 etc., Huo Zheke
Temporarily to store oneself data (for example, telephone directory, message, still image, video etc.) through exporting or will export.And
And, memory 160 can store the vibration of various modes on being exported when touching and being applied to touch-screen and audio signal
Data.
Memory 160 can include the storage medium of at least one type, and the storage medium includes flash memory, hard disk, many
Media card, card-type memory (for example, SD or DX memories etc.), random access storage device (RAM), static random-access storage
Device (SRAM), read-only storage (ROM), Electrically Erasable Read Only Memory (EEPROM), programmable read only memory
(PROM), magnetic storage, disk, CD etc..And, mobile terminal 1 00 can perform memory with by network connection
The network storage device cooperation of 160 store function.
The overall operation of the generally control mobile terminal of controller 180.For example, controller 180 is performed and voice call, data
Communication, video calling etc. related control and treatment.In addition, controller 180 can be included for reproducing (or playback) many matchmakers
The multi-media module 181 of volume data, multi-media module 181 can be constructed in controller 180, or can be structured as and control
Device 180 is separated.Controller 180 can be with execution pattern identifying processing, the handwriting input that will be performed on the touchscreen or picture
Draw input and be identified as character or image.
Power subsystem 190 receives external power or internal power under the control of controller 180 and provides operation each unit
Appropriate electric power needed for part and component.
Various implementation methods described herein can be with use such as computer software, hardware or its any combination of calculating
Machine computer-readable recording medium is implemented.Implement for hardware, implementation method described herein can be by using application-specific IC
(ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), scene can
Programming gate array (FPGA), processor, controller, microcontroller, microprocessor, it is designed to perform function described herein
At least one in electronic unit is implemented, and in some cases, such implementation method can be implemented in controller 180.
For software implementation, the implementation method of such as process or function can with allow to perform the single of at least one function or operation
Software module is implemented.Software code can be come by the software application (or program) write with any appropriate programming language
Implement, software code can be stored in memory 160 and performed by controller 180.
So far, oneself according to its function through describing mobile terminal 1 00.In addition, the mobile terminal 1 00 in the embodiment of the present invention
Can be such as folded form, board-type, oscillating-type, sliding-type and other various types of mobile terminals, not do herein specifically
Limit.
Mobile terminal 1 00 as shown in Figure 1 may be constructed such that using via frame or packet transmission data it is all if any
Line and wireless communication system and satellite-based communication system are operated.
The communication system that mobile terminal wherein of the invention can be operated is described referring now to Fig. 2.
Such communication system can use different air interface and/or physical layer.For example, used by communication system
Air interface includes such as frequency division multiple access (FDMA), time division multiple acess (TDMA), CDMA (CDMA) and universal mobile communications system
System (UMTS) (especially, Long Term Evolution (LTE)), global system for mobile communications (GSM) etc..As non-limiting example, under
The description in face is related to cdma communication system, but such teaching is equally applicable to other types of system.
With reference to Fig. 2, cdma wireless communication system can include multiple mobile terminal 1s 00, multiple base station (BS) 270, base station
Controller (BSC) 275 and mobile switching centre (MSC) 280.MSC 280 is configured to and Public Switched Telephony Network (PSTN)
290 form interface.MSC 280 is also structured to be formed with the BSC 275 that can be couple to base station 270 via back haul link and connects
Mouthful.If any one in the interface that back haul link can be known according to Ganji is constructed, the interface can include such as Europe mark
Quasi- high power capacity digital circuit/Unite States Standard high power capacity digital circuit (E1/T1), asynchronous transfer mode (ATM), procotol
(IP), point-to-point protocol (PPP), frame relay, high-bit-rate digital subscriber line road (HDSL), Asymmetrical Digital Subscriber Line (ADSL)
Or all kinds digital subscriber line (xDSL).It will be appreciated that system can include multiple BSC 275 as shown in Figure 2.
Each BS 270 can service one or more subregions (or region), by multidirectional antenna or the day of sensing specific direction
Each subregion of line covering is radially away from BS 270.Or, each subregion can by two for diversity reception or more
Multiple antennas are covered.Each BS 270 may be constructed such that the multiple frequency distribution of support, and the distribution of each frequency has specific frequency
Spectrum (for example, 1.25MHz, 5MHz etc.).
What subregion and frequency were distributed intersects can be referred to as CDMA Channel.BS 270 can also be referred to as base station transceiver
System (BTS) or other equivalent terms.In this case, term " base station " can be used for broadly representing single BSC
275 and at least one BS 270.Base station can also be referred to as " cellular station ".Or, each subregion of specific BS 270 can be claimed
It is multiple cellular stations.
As shown in Figure 2, broadcast singal is sent to broadcsting transmitter (BT) 295 mobile terminal operated in system
100.Broadcasting reception module 111 as shown in Figure 1 is arranged at mobile terminal 1 00 to receive the broadcast sent by BT 295
Signal.In fig. 2 it is shown that several global positioning system (GPS) satellites 300.Satellite 300 helps position multiple mobile terminals
At least one of 100.
In fig. 2, multiple satellites 300 are depicted, it is understood that be, it is possible to use any number of satellite obtains useful
Location information.Location information module 115 as shown in Figure 1 is (such as:GPS) it is generally configured to coordinate to obtain with satellite 300
The location information that must be wanted.Substitute GPS tracking techniques or outside GPS tracking techniques, it is possible to use can track mobile whole
Other technologies of the position at end.In addition, at least one gps satellite 300 can optionally or additionally process satellite dmb biography
It is defeated.
Used as a typical operation of wireless communication system, BS 270 receives the reverse strand from various mobile terminal 1s 00
Road signal.Mobile terminal 1 00 generally participates in call, information receiving and transmitting and other types of communication.Each of certain base station reception is anti-
Processed in specific BS 270 to link signal.The data of acquisition are forwarded to the BSC 275 of correlation.BSC provides logical
Words resource allocation and the mobile management function of the coordination including the soft switching process between BS 270.BSC 275 will also be received
Data be routed to MSC 280, its provide for PSTN 290 formed interface extra route service.Similarly, PSTN
290 form interface with MSC 280, and MSC and BSC 275 form interface, and BSC 275 correspondingly controls BS 270 with by forward direction
Link signal is sent to mobile terminal 1 00.
Based on above-mentioned mobile terminal hardware configuration and communication system, the application method each embodiment is proposed.
As shown in figure 3, the embodiment of the present invention proposes a kind of method for realizing interactive image segmentation, including:
S310, after detecting the smearing track on original image, geometric figure of the construction comprising the smearing track is simultaneously right
The geometric figure is extended to form mark zone, it is determined that one piece of region comprising the smearing track is used as area-of-interest,
The area-of-interest includes the mark zone;Generate the input mask artwork of image segmentation algorithm:To own in the mark zone
Pixel as the foreground point in the mask artwork, using the pixel outside mark zone in the area-of-interest as the mask artwork
In background dot;
S320, obtains the depth map comprising destination object depth information, according to the depth map and mask artwork determine
The partitioning parameters of each pixel on mask artwork, the partitioning parameters are used to represent that pixel judgement is foreground point or the probability of background dot
And the depth value difference of the pixel and adjacent pixel;
S330, builds non-directed graph, and the partitioning parameters of each pixel in the mask artwork are mapped into the non-directed graph
In, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask artwork after fine segmentation;
S340, the mask artwork after the fine segmentation is partitioned into from the original image that the destination object is obtained is shot
The corresponding image in middle foreground point;
Methods described can also include following features:
Wherein, smearing is a kind of mode for marking destination object, and smearing is marked in the interior zone of destination object;
Wherein, mask artwork refers to that the part or all of pixel of a sub-picture has been carried out to be generated after prospect is distinguished with background
Mark figure, each pixel on the mask artwork is marked as foreground point or background dot.
In the present embodiment, described image partitioning algorithm is GrabCut algorithms.
In one embodiment, it is described to construct the geometric figure comprising the smearing track and the geometric figure is entered
Row extension forms mark zone, it is determined that one piece of region comprising the smearing track is used as area-of-interest, including:
The minimum enclosed rectangle of the smearing track is constructed, the length L according to the smearing track is external to the minimum
Rectangle is extended to form mark zone;
For geometry template is applied mechanically in the smearing track, the smearing rail is included according to the geometry template generation
The area-of-interest (Region of Interest, ROI) of mark, if the area-of-interest does not include the mark zone,
Then the area-of-interest is extended to comprising the mark zone;Or expansion process is carried out to the mark zone form interested
Region.
Wherein, the geometric figure includes:Rectangle, polygon, ellipse etc..
Wherein, the geometry template includes:Rectangle template, polygon splat or oval template;
Wherein, the minimum enclosed rectangle of the construction smearing track, the length L according to the smearing track is to described
Minimum enclosed rectangle is extended to form mark zone, including:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum horizontal stroke
Coordinate value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, four coordinates on summit of the minimum enclosed rectangle are successively
It is:(x_min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark
Four apex coordinates in area are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a),
(x_max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants;B can be empirical value.
In the present embodiment, the segmentation ginseng of each pixel on the mask artwork is determined according to the depth map and mask artwork
Number, including:
Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps;Iteration runs E
Step and M steps, the iterative process is stopped after iteration operation reaches the condition of convergence;Last time is performed into M steps
The classification of the pixel for obtaining is defined as the classification of the pixel, and the pixel that last time performs the acquisition of M steps is returned
Belong to the most probable value P of certain clustermaxIt is defined as the area item partitioning parameters of the pixel, the area item partitioning parameters are
The pixel judgement is foreground point or the probability of background dot;
Wherein, the E steps and M steps include following treatment respectively:
E steps:The position relationship between depth value and pixel according to each pixel on the mask artwork is by clustering
Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined;Wherein, the classification of pixel includes prospect
Point or background dot;The classification of cluster includes foreground point cluster or background dot cluster;
M steps:GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one
Pixel, the most probable value P according to the pixelmaxCorresponding cluster determines the classification of the pixel;
Alternatively, the condition of convergence of the EM methods can be:Stop iteration when the number of times of iteration operation reaches threshold value
Process;
In the present embodiment, the segmentation ginseng of each pixel on the mask artwork is determined according to the depth map and mask artwork
Number, also includes:Depth value difference according to the pixel and adjacent pixel determines the border item partitioning parameters of the pixel;
Wherein, to any one pixel, by the depth value absolute difference between the pixel and each neighbouring pixel
Added up, then to adding up and being normalized, cumulative and as the pixel the border item point after being normalized
Cut parameter;
Alternatively, each pixel neighbouring with pixel can be 8 pixels on the pixel periphery;
In the present embodiment, non-directed graph is built, the partitioning parameters of each pixel in the mask artwork is mapped to institute
In stating non-directed graph, including:
A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph0And Q1, the hanging point Q0For
Virtual foreground point, the hanging point Q1It is virtual background point;Each is set up on the mask artwork in the plane of the non-directed graph
The mapping point of pixel, in mapping point and the hanging point Q of foreground point0Between set up line, mapping point and institute in background dot
State hanging point Q1Between set up line;
To any one pixel P in mask artworki, by the pixel PiArea item partitioning parameters as the non-directed graph
Middle mapping point Pi' weights, by the pixel PiBorder item partitioning parameters as mapping point P in the non-directed graphi' with it is hanging
Point Q0Or Q1Between line weights;
In the present embodiment, it is described the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain fine point
Mask artwork after cutting, including:
Iteration performs following step C and D steps, and iterative process is stopped after iteration operation reaches the condition of convergence, will be described
Each pixel in prospect point set Q is used as the foreground point in the mask artwork after fine segmentation;
Wherein, step C and D steps include following treatment respectively:
Step C:One part of pixel in non-directed graph is divided into and vacantly point Q0Similar foreground point, by being divided into prospect
The pixel of point constitutes prospect point set Q;
D steps:The weights summation of the prospect point set Q is calculated, the weights summation is institute in the prospect point set Q
Have foreground point weights and, along with all foreground points in the prospect point set Q and hanging point Q0Between line weights and;
Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability;
Alternatively, the original image that the shooting destination object is obtained can be cromogram, artwork master or infrared figure.
In one embodiment, before the input mask artwork of generation image segmentation algorithm, methods described also includes:
If currently detected smearing track is directed to the first time mark of the original image, will be currently detected
The mark zone determined of smearing track as final mark zone, by currently detected smearing track determine it is interested
Region is used as final area-of-interest;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
Whether have between the mark zone that the smearing track that the mark zone and last time that the smearing track for measuring is determined are detected is determined
Occur simultaneously, if occuring simultaneously, then the smearing rail that the mark zone determined currently detected smearing track is detected with the last time
The mark zone that mark is determined is merged, using the mark zone after merging as final mark zone;If do not occured simultaneously, ought
Before the mark zone determined of the smearing track that detects as final mark zone;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
The area-of-interest that area-of-interest that the smearing track for measuring is determined is determined with the last smearing track for detecting it
Between whether have common factor, if occuring simultaneously, then the area-of-interest determined currently detected smearings track and last inspection
The area-of-interest that the smearing track for measuring is determined is merged, using the area-of-interest after merging as final interested
Region;If do not occured simultaneously, the area-of-interest that currently detected smearing track is determined is used as final interested
Region.
Wherein, in addition to first time smears, the corresponding mark zone of smearing each time afterwards and area-of-interest all with it is upper
Once smear the mark zone determined of track and area-of-interest be compared so that foreground point in input mask artwork and
Background dot is more accurate, improves the quality of segmentation figure picture.
In correlation technique, the foreground point in the input mask artwork of image segmentation algorithm is user's hand labeled, original graph
As behind upper removing foreground point other pixel mark be have that foreground point mark is less and input mask artwork size compared with
It is big to cause image segmentation algorithm to be to distinguish foreground point and the iterations of background dot to increase, Riming time of algorithm problem long.
After using the method for the embodiment of the present invention, mark zone is generated by constructive geometry image, by the element marking in the mark zone
It is foreground point, so that the prospect for being marked in the input mask artwork of expansion image segmentation algorithm automatically is counted out, it is emerging by generating sense
Interesting region with substitute view picture original image can reduce image segmentation algorithm mark background count out, can reduce image segmentation calculation
Method is to distinguish the iterations of foreground point and background dot, significantly decreases the run time of image segmentation algorithm.On the other hand,
The technical scheme of the embodiment of the present invention using depth map calculate partitioning parameters, when the destination object to be extracted color character not
When substantially, the effect of image segmentation can be improved.
As shown in figure 4, the embodiment of the present invention proposes a kind of device for realizing interactive image segmentation, including:
Pretreatment module 401, after detecting the smearing track on original image, construction includes the smearing track
Geometric figure simultaneously is extended to form mark zone to the geometric figure, it is determined that one piece of region conduct comprising the smearing track
Area-of-interest, the area-of-interest includes the mark zone;Generate the input mask artwork of image segmentation algorithm:By the mark
All of pixel is used as the foreground point in the mask artwork in note area, using the pixel outside mark zone in the area-of-interest as
Background dot in the mask artwork;
Partitioning parameters computing module 402, for obtaining the depth map comprising destination object depth information, according to the depth
Figure and mask artwork determine the partitioning parameters of each pixel on the mask artwork, and the partitioning parameters are used to represent that pixel judgement is preceding
The probability and the pixel of sight spot or background dot and the depth value difference of adjacent pixel;
Mask artwork adjusting module 403, for building non-directed graph, by the partitioning parameters of each pixel in the mask artwork
It is mapped in the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, after obtains fine segmentation
Mask artwork;
Output module 404, for being partitioned into described fine point from the original image that the destination object is obtained is shot
The corresponding image in foreground point in mask artwork after cutting;
Described device can also include following features:
Wherein, smearing is a kind of mode for marking destination object, and smearing is marked in the interior zone of destination object;
Wherein, mask artwork refers to that the part or all of pixel of a sub-picture has been carried out to be generated after prospect is distinguished with background
Mark figure, each pixel on the mask artwork is marked as foreground point or background dot.
In the present embodiment, described image partitioning algorithm is GrabCut algorithms.
In one embodiment, pretreatment module, for constructing in the following ways comprising the several of the smearing track
What figure simultaneously is extended to form mark zone to the geometric figure, it is determined that one piece of region comprising the smearing track is used as sense
Interest region:
The minimum enclosed rectangle of the smearing track is constructed, the length L according to the smearing track is external to the minimum
Rectangle is extended to form mark zone;
For geometry template is applied mechanically in the smearing track, the smearing rail is included according to the geometry template generation
The area-of-interest of mark, if the area-of-interest does not include the mark zone, the area-of-interest is extended to
Comprising the mark zone;Or expansion process formation area-of-interest is carried out to the mark zone.
Wherein, the geometric figure includes:Rectangle, polygon, ellipse etc..
Wherein, the geometry template includes:Rectangle template, polygon splat or oval template;
Wherein, the minimum enclosed rectangle of the construction smearing track, the length L according to the smearing track is to described
Minimum enclosed rectangle is extended to form mark zone, including:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum horizontal stroke
Coordinate value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, four coordinates on summit of the minimum enclosed rectangle are successively
It is:(x_min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark
Four apex coordinates in area are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a),
(x_max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants;B can be empirical value.
In the present embodiment, partitioning parameters computing module, in the following ways according to the depth map and mask artwork
Determine the partitioning parameters of each pixel on the mask artwork:
Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps;Iteration runs E
Step and M steps, the iterative process is stopped after iteration operation reaches the condition of convergence;Last time is performed into M steps
The classification of the pixel for obtaining is defined as the classification of the pixel, and the pixel that last time performs the acquisition of M steps is returned
Belong to the most probable value P of certain clustermaxIt is defined as the area item partitioning parameters of the pixel, the area item partitioning parameters are
The pixel judgement is foreground point or the probability of background dot;
Wherein, the E steps and M steps include following treatment respectively:
E steps:The position relationship between depth value and pixel according to each pixel on the mask artwork is by clustering
Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined;Wherein, the classification of pixel includes prospect
Point or background dot;The classification of cluster includes foreground point cluster or background dot cluster;
M steps:GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one
Pixel, the most probable value P according to the pixelmaxCorresponding cluster determines the classification of the pixel;
Alternatively, the condition of convergence of the EM methods can be:Stop iteration when the number of times of iteration operation reaches threshold value
Process;
In the present embodiment, partitioning parameters computing module, is additionally operable in the following ways according to the depth map and mask
Figure determines the partitioning parameters of each pixel on the mask artwork:Depth value difference according to the pixel and adjacent pixel determines institute
State the border item partitioning parameters of pixel;
Wherein, to any one pixel, by the depth value absolute difference between the pixel and each neighbouring pixel
Added up, then to adding up and being normalized, cumulative and as the pixel the border item point after being normalized
Cut parameter;
Alternatively, each pixel neighbouring with pixel can be 8 pixels on the pixel periphery.
In the present embodiment, mask artwork adjusting module, for building non-directed graph in the following ways and by the mask artwork
In the partitioning parameters of each pixel be mapped in the non-directed graph:
A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph0And Q1, the hanging point Q0For
Virtual foreground point, the hanging point Q1It is virtual background point;Each is set up on the mask artwork in the plane of the non-directed graph
The mapping point of pixel, in mapping point and the hanging point Q of foreground point0Between set up line, mapping point and institute in background dot
State hanging point Q1Between set up line;
To any one pixel P in mask artworki, by the pixel PiArea item partitioning parameters as the non-directed graph
Middle mapping point Pi' weights, by the pixel PiBorder item partitioning parameters as mapping point P in the non-directed graphi' with it is hanging
Point Q0Or Q1Between line weights.
In the present embodiment, mask artwork adjusting module, in the following ways according to minimal cut-maximum-flow algorithm to institute
State non-directed graph to be processed, obtain the mask artwork after fine segmentation:
Iteration performs following step C and D steps, and iterative process is stopped after iteration operation reaches the condition of convergence, will be described
Each pixel in prospect point set Q is used as the foreground point in the mask artwork after fine segmentation;
Wherein, step C and D steps include following treatment respectively:
Step C:One part of pixel in non-directed graph is divided into and vacantly point Q0Similar foreground point, by being divided into prospect
The pixel of point constitutes prospect point set Q;
D steps:The weights summation of the prospect point set Q is calculated, the weights summation is institute in the prospect point set Q
Have foreground point weights and, along with all foreground points in the prospect point set Q and hanging point Q0Between line weights and;
Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability.
Alternatively, the original image that the shooting destination object is obtained is cromogram, artwork master or infrared figure.
In one embodiment, pretreatment module, was additionally operable to before the input mask artwork of generation image segmentation algorithm,
It is marked the adjustment of area and area-of-interest:
If currently detected smearing track is directed to the first time mark of the original image, will be currently detected
The mark zone determined of smearing track as final mark zone, by currently detected smearing track determine it is interested
Region is used as final area-of-interest;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
Whether have between the mark zone that the smearing track that the mark zone and last time that the smearing track for measuring is determined are detected is determined
Occur simultaneously, if occuring simultaneously, then the smearing rail that the mark zone determined currently detected smearing track is detected with the last time
The mark zone that mark is determined is merged, using the mark zone after merging as final mark zone;If do not occured simultaneously, ought
Before the mark zone determined of the smearing track that detects as final mark zone;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
The area-of-interest that area-of-interest that the smearing track for measuring is determined is determined with the last smearing track for detecting it
Between whether have common factor, if occuring simultaneously, then the area-of-interest determined currently detected smearings track and last inspection
The area-of-interest that the smearing track for measuring is determined is merged, using the area-of-interest after merging as final interested
Region;If do not occured simultaneously, the area-of-interest that currently detected smearing track is determined is used as final interested
Region.
The method of the embodiment of the present invention, generates mark zone, by the pixel mark in the mark zone by constructive geometry image
Foreground point is designated as, so that the prospect for being marked in the input mask artwork of expansion image segmentation algorithm automatically is counted out, is felt by generating
Interest region with substitute view picture original image can reduce image segmentation algorithm mark background count out, image segmentation can be reduced
Algorithm is the iterations for distinguishing foreground point and background dot, significantly decreases the run time of image segmentation algorithm.The opposing party
Face, the technical scheme of the embodiment of the present invention calculates partitioning parameters using depth map, when the color character of the destination object to be extracted
When unobvious, the effect of image segmentation can be improved.
Embodiment 3
The embodiment of the present invention also provides a kind of terminal, and the terminal includes the above-mentioned device for realizing interactive image segmentation.
Using example 1
User's destination object interested in oneself on the original image is smeared, using the preprocess method of this paper
Image is pre-processed, be may comprise steps of:
Step S501, detects user's selection destination object is marked by the way of smearing;
Such as, two buttons for being used to mark are provided on interface, one is " smearing ", and one is " sketching the contours ", if user
" smearing " button is clicked, has then been pre-processed to smearing track.
Wherein, it is two kinds of different modes for marking destination object to smear and sketch the contours;
Usually, smearing is marked in the interior zone of destination object, and it is along the outer wheels of destination object to sketch the contours
Exterior feature is marked;
Step S502, detects user and is smeared on the original image;
Such as, as shown in Fig. 5-a, user is smeared on the original image, and destination object is " stapler ", wherein,
Original image is cromogram;
Step S503, constructs the minimum enclosed rectangle of the smearing track, and the length L according to the smearing track is to described
Minimum enclosed rectangle is extended to form mark zone;For rectangle template is applied mechanically in the smearing track, given birth to according to the rectangle template
Into the area-of-interest comprising the smearing track;If the area-of-interest does not include the mark zone, will be described
Area-of-interest is extended to comprising the mark zone;
Wherein, the minimum enclosed rectangle of the construction smearing track, the length L according to the smearing track is to described
Minimum enclosed rectangle is extended to form mark zone, including:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum horizontal stroke
Coordinate value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, four coordinates on summit of the minimum enclosed rectangle are successively
It is:(x_min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark
Four apex coordinates in area are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a),
(x_max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants;
Such as, as shown in Fig. 5-b, it is first rectangle region comprising the smearing track to smear the corresponding mark zone in track
Domain, area-of-interest is second rectangular area comprising the mark zone, and the frame of the area-of-interest is represented by dashed line.
Step S504, the input mask artwork of generation image segmentation algorithm (GrabCut algorithms):To own in the mark zone
Pixel as the foreground point in mask artwork, using the pixel outside mark zone in the area-of-interest as the background in mask artwork
Point.
Such as, as shown in Fig. 5-c, the rectangle darker regions comprising smearing track are mark zones, and the mark zone is mask
Foreground point block in figure (input mask artwork);The edge of area-of-interest is indicated with dotted line frame, is removed in the dotted line frame
The part for going to mark zone (foreground point block) is the background dot block in mask artwork.
Step S505, obtains the depth map comprising destination object depth information;
As shown in Fig. 5-d, depth map is a figure comprising depth information, and the size with cromogram is consistent;Depth
In figure, farther out, the shallower part shooting distance of color is nearer for the deeper part shooting distance of color.
Step S506, the partitioning parameters of each pixel on the mask artwork, institute are determined according to the depth map and mask artwork
It is the probability and the pixel of foreground point or background dot and the depth of adjacent pixel that partitioning parameters are stated for representing pixel judgement
Value difference is different;
Wherein, to any one pixel on the mask artwork, the partitioning parameters of the pixel include area item partitioning parameters
With border item partitioning parameters;It is general for foreground point or background dot that the area item partitioning parameters of the pixel refer to pixel judgement
Rate;The border item partitioning parameters of the pixel refer to the depth value difference of the pixel and adjacent pixel;
Wherein, the area item partitioning parameters of each pixel on the mask artwork are determined in the following ways:
Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps;Iteration runs E
Step and M steps, the iterative process is stopped after the number of times of iteration operation reaches predetermined number of times;Last time is performed
The classification of the pixel that M steps are obtained is defined as the classification of the pixel, and last time is performed into the picture that M steps are obtained
Element belongs to the most probable value P of certain clustermaxIt is defined as the area item partitioning parameters of the pixel, the area item segmentation ginseng
Number is that the pixel judgement is foreground point or the probability of background dot;
Wherein, the E steps and M steps include following treatment respectively:
E steps:The position relationship between depth value and pixel according to each pixel on the mask artwork is by clustering
Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined;Wherein, the classification of pixel includes prospect
Point or background dot;The classification of cluster includes foreground point cluster or background dot cluster;
M steps:GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one
Pixel, the most probable value P according to the pixelmaxCorresponding cluster determines the classification of the pixel;
Wherein, the border item partitioning parameters of each pixel on the mask artwork are determined in the following ways:To any one
Pixel, the depth value absolute difference between each pixel in the pixel and 8 neighbouring neighborhoods is added up, then to tired
Plus and be normalized, adding up and as the border item partitioning parameters of the pixel after being normalized;
Step S507, builds non-directed graph, and the partitioning parameters of each pixel in the mask artwork are mapped into the nothing
In to figure, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask artwork after fine segmentation;
Wherein, non-directed graph sets two hanging point Q as shown in Fig. 5-e outside the plane of the non-directed graph0And Q1, it is described outstanding
Null point Q0It is virtual foreground point, the hanging point Q1It is virtual background point;The mask artwork is set up in the plane of the non-directed graph
The mapping point of upper each pixel, in mapping point and the hanging point Q of foreground point0Between set up line, in the mapping of background dot
Point and the hanging point Q1Between set up line;
To any one pixel P in mask artworki, by the pixel PiArea item partitioning parameters as the non-directed graph
Middle mapping point Pi' weights, by the pixel PiBorder item partitioning parameters as mapping point P in the non-directed graphi' with it is hanging
Point Q0Or Q1Between line weights.
Wherein, minimal cut MinCut- max-flow MaxFlow algorithms are run according to the non-directed graph in the following ways, is obtained
Obtain the mask artwork after fine segmentation:
Iteration performs following step C and D steps, and iterative process is stopped after iteration operation reaches the condition of convergence, will be described
Each pixel in prospect point set Q is used as the foreground point in the mask artwork after fine segmentation;
Wherein, step C and D steps include following treatment respectively:
Step C:One part of pixel in non-directed graph is divided into and vacantly point Q0Similar foreground point, by being divided into prospect
The pixel of point constitutes prospect point set Q;
D steps:The weights summation of the prospect point set Q is calculated, the weights summation is institute in the prospect point set Q
Have foreground point weights and, along with all foreground points in the prospect point set Q and hanging point Q0Between line weights and;
Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability;
Wherein, the mask artwork after fine segmentation is as shown in Fig. 5-f, comprising smearing track and with target image (stapler)
The irregular darker regions of profile are mark zones, and the mark zone is the foreground point block in the mask artwork after fine segmentation;Sense
The part that mark zone (foreground point block) is removed in interest region is the background dot block in mask artwork.Mask after fine segmentation
Figure is compared with initial mask artwork, and the border between foreground point block and background dot block is more careful.
Step S508, is partitioned into covering after the fine segmentation from the original image that the destination object is obtained is shot
The corresponding image in foreground point in mould figure;
Wherein, destination object is partitioned into from the original cromogram according to the mask artwork after fine segmentation, is partitioned into
" stapler " image as shown in Fig. 5-g.
The method of this application example, the first rectangle neighboring region being extended to by will smear track is defined as mark zone,
All pixels in mark zone are labeled as foreground point, are marked in the input mask artwork that can automatically expand image segmentation algorithm
Prospect is counted out, by using the second rectangular area comprising the smearing track as area-of-interest substituting view picture original graph
As the background that can reduce image segmentation algorithm mark is counted out, image segmentation algorithm can be reduced to distinguish foreground point and background
The iterations of point, significantly decreases the run time of image segmentation algorithm.On the other hand, the technical scheme of the embodiment of the present invention
Partitioning parameters are calculated using depth map, when the color character of the destination object to be extracted is not obvious, image point can be improved
The effect cut.
Using example 2
User's destination object interested in oneself on the original image is smeared, using the preprocess method of this paper
Image is pre-processed, many inputs (repeatedly smearing) can be received, comprised the following steps:
Step S601, detects user's selection destination object is marked by the way of smearing;
Such as, two buttons for being used to mark are provided on interface, one is " smearing ", and one is " sketching the contours ", if user
" smearing " button is clicked, has then been pre-processed to smearing track.
Step S602, detects user and is smeared on the original image;
Such as, as shown in Fig. 6-a-1, user has carried out smearing for the first time on the original image, and destination object is " stapler
Machine ";
Such as, as shown in Fig. 6-a-2, user has carried out second and has smeared on the original image, and destination object is " stapler
Machine ";
Step S603, constructs the minimum enclosed rectangle of the smearing track, and the length L according to the smearing track is to described
Minimum enclosed rectangle is extended to form mark zone;For rectangle template is applied mechanically in the smearing track, given birth to according to the rectangle template
Into the area-of-interest comprising the smearing track;If the area-of-interest does not include the mark zone, will be described
Area-of-interest is extended to comprising the mark zone;
Wherein, the minimum enclosed rectangle of the construction smearing track, the length L according to the smearing track is to described
Minimum enclosed rectangle is extended to form mark zone, including:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum horizontal stroke
Coordinate value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, four coordinates on summit of the minimum enclosed rectangle are successively
It is:(x_min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark
Four apex coordinates in area are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a),
(x_max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants;
Such as, as shown in Fig. 6-b-1, it is to smear track comprising the first time that the corresponding mark zone in track is smeared for the first time
First rectangular area, area-of-interest is second rectangular area comprising the mark zone, the frame of area-of-interest
It is represented by dashed line;
Such as, as shown in Fig. 6-b-2, it is to smear track comprising described second that second is smeared the corresponding mark zone in track
First rectangular area, area-of-interest is second rectangular area comprising the mark zone, the frame of area-of-interest
It is represented by dashed line;
Step S604, generates the input mask artwork of image segmentation algorithm, including:
If currently detected smearing track is directed to the first time mark of the original image, will be currently detected
The mark zone determined of smearing track as final mark zone, by currently detected smearing track determine it is interested
Region is used as final area-of-interest;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
Whether have between the mark zone that the smearing track that the mark zone and last time that the smearing track for measuring is determined are detected is determined
Occur simultaneously, if occuring simultaneously, then the smearing rail that the mark zone determined currently detected smearing track is detected with the last time
The mark zone that mark is determined is merged, using the mark zone after merging as final mark zone;If do not occured simultaneously, ought
Before the mark zone determined of the smearing track that detects as final mark zone;
If currently detected smearing track is not directed to the first time mark of the original image, current inspection is judged
The area-of-interest that area-of-interest that the smearing track for measuring is determined is determined with the last smearing track for detecting it
Between whether have common factor, if occuring simultaneously, then the area-of-interest determined currently detected smearings track and last inspection
The area-of-interest that the smearing track for measuring is determined is merged, using the area-of-interest after merging as final interested
Region;If do not occured simultaneously, the area-of-interest that currently detected smearing track is determined is used as final interested
Region;
Using all of pixel in the mark zone as the foreground point in the mask artwork, by area-of-interest acceptance of the bid
Pixel outside note area is used as the background dot in the mask artwork;
Such as, as shown in Fig. 6-c-1, the rectangle darker regions that track is smeared comprising first time are mark zones, the mark
Area is the foreground point block in mask artwork (input mask artwork);The edge of area-of-interest is indicated with dotted line frame, the void
The part that mark zone (foreground point block) is removed in wire frame is the background dot block in mask artwork.
Such as, as shown in Fig. 6-c-2, the first time smears the corresponding mark zone in track and smears track with described second
There is common factor between corresponding mark zone, then the union of final mark zone Shi Liangge mark zones;The first time smears track pair
There is common factor between the area-of-interest answered area-of-interest corresponding with second smearing track, then final region of interest
Domain is two unions of area-of-interest.The mark zone is the foreground point block in mask artwork (input mask artwork);It is interested
The edge in region is indicated with dotted line frame, during the part that mark zone (foreground point block) is removed in the dotted line frame is mask artwork
Background dot block.
Step S605, obtains the depth map comprising destination object depth information;
Step S606, the partitioning parameters of each pixel on the mask artwork, institute are determined according to the depth map and mask artwork
It is the probability and the pixel of foreground point or background dot and the depth of adjacent pixel that partitioning parameters are stated for representing pixel judgement
Value difference is different;
Step S607, builds non-directed graph, and the partitioning parameters of each pixel in the mask artwork are mapped into the nothing
In to figure, minimal cut MinCut- max-flow MaxFlow algorithms are run according to the non-directed graph, obtain the mask after fine segmentation
Figure;
Step S608, is partitioned into covering after the fine segmentation from the original image that the destination object is obtained is shot
The corresponding image in foreground point in mould figure;
Wherein, the image of " stapler " is partitioned into from the original cromogram according to the mask artwork after fine segmentation.
Wherein, in addition to first time smears, the corresponding mark zone of smearing each time afterwards and area-of-interest all with it is upper
Once smear the mark zone determined of track and area-of-interest be compared so that foreground point in input mask artwork and
Background dot is more accurate, improves the quality of segmentation figure picture;
The method of this application example, the first rectangle neighboring region being extended to by will smear track is defined as mark zone,
All pixels in mark zone are labeled as foreground point, are marked in the input mask artwork that can automatically expand image segmentation algorithm
Prospect is counted out, by using the second rectangular area comprising the smearing track as area-of-interest substituting view picture original graph
As the background that can reduce image segmentation algorithm mark is counted out, image segmentation algorithm can be reduced to distinguish foreground point and background
The iterations of point, significantly decreases the run time of image segmentation algorithm.On the other hand, the technical scheme of the embodiment of the present invention
Partitioning parameters are calculated using depth map, when the color character of the destination object to be extracted is not obvious, image point can be improved
The effect cut.
It should be noted that herein, term " including ", "comprising" or its any other variant be intended to non-row
His property is included, so that process, method, article or device including a series of key elements not only include those key elements, and
And also include other key elements being not expressly set out, or also include for this process, method, article or device institute are intrinsic
Key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including institute
Also there is other identical element in process, method, article or the device of stating key element.
The embodiments of the present invention are for illustration only, and the quality of embodiment is not represented.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases
The former is more preferably implementation method.Based on such understanding, technical scheme is substantially done to prior art in other words
The part for going out contribution can be embodied in the form of software product, and the computer software product storage is in a storage medium
In (such as ROM/RAM, magnetic disc, CD), including some instructions are used to so that a station terminal equipment (can be mobile phone, computer, clothes
Business device, air-conditioner, or network equipment etc.) perform method described in each embodiment of the invention.
The preferred embodiments of the present invention are these are only, the scope of the claims of the invention is not thereby limited, it is every to utilize this hair
Equivalent structure or equivalent flow conversion that bright specification and accompanying drawing content are made, or directly or indirectly it is used in other related skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of method for realizing interactive image segmentation, including:
After detecting the smearing track on original image, construction includes the geometric figure of the smearing track and to the geometric graph
Shape is extended to form mark zone, it is determined that one piece of region comprising the smearing track is used as area-of-interest, it is described interested
Region includes the mark zone;Generate the input mask artwork of image segmentation algorithm:Using all of pixel in the mark zone as
Foreground point in the mask artwork, using the pixel outside mark zone in the area-of-interest as the background in the mask artwork
Point;
The depth map comprising destination object depth information is obtained, is determined according to the depth map and mask artwork each on the mask artwork
The partitioning parameters of individual pixel, the partitioning parameters are used to represent that pixel judgement is foreground point or the probability and the picture of background dot
The depth value difference of element and adjacent pixel;
Non-directed graph is built, the partitioning parameters of each pixel in the mask artwork is mapped in the non-directed graph, according to most
It is small cut-maximum-flow algorithm processed the non-directed graph, obtains the mask artwork after fine segmentation;
Foreground point in the mask artwork after the fine segmentation is partitioned into from the original image that the destination object is obtained is shot
Corresponding image.
2. method according to claim 1, it is characterised in that:
The partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, including:Enter by EM methods
Row gauss hybrid models GMM is calculated, and the EM methods include E steps and M steps;Iteration runs E steps and M steps, described
Iteration operation stops the iterative process after reaching the condition of convergence;Last time is performed into dividing for the pixel that M steps are obtained
Class is defined as the classification of the pixel, and the pixel that last time performs the acquisition of M steps is belonged into the most general of certain cluster
Rate value PmaxIt is defined as the area item partitioning parameters of the pixel, the area item partitioning parameters are that the pixel judgement is prospect
The probability of point or background dot;
Wherein, the E steps and M steps include following treatment respectively:
E steps:The position relationship between depth value and pixel according to each pixel on the mask artwork will be same by clustering
The pixel of type is polymerized to one or more clusters, determines the GMM model of each cluster;Wherein, the classification of pixel include foreground point or
Background dot;The classification of cluster includes foreground point cluster or background dot cluster;
M steps:GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one pixel,
Most probable value P according to the pixelmaxCorresponding cluster determines the classification of the pixel.
3. method according to claim 2, it is characterised in that:
The partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, is also included:According to described
Pixel determines the border item partitioning parameters of the pixel with the depth value difference of adjacent pixel;
Wherein, to any one pixel, the depth value absolute difference between the pixel and each neighbouring pixel is carried out
It is cumulative, then to adding up and being normalized, cumulative and as the pixel the border item after being normalized splits ginseng
Number.
4. method according to claim 3, it is characterised in that:
Non-directed graph is built, the partitioning parameters of each pixel in the mask artwork are mapped in the non-directed graph, including:
A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph0And Q1, the hanging point Q0For virtual
Foreground point, the hanging point Q1It is virtual background point;Each pixel on the mask artwork is set up in the plane of the non-directed graph
Mapping point, in mapping point and the hanging point Q of foreground point0Between set up line, it is outstanding with described in the mapping point of background dot
Null point Q1Between set up line;
To any one pixel P in mask artworki, by the pixel PiArea item partitioning parameters as being reflected in the non-directed graph
Exit point Pi' weights, by the pixel PiBorder item partitioning parameters as mapping point P in the non-directed graphi' with hanging point Q0
Or Q1Between line weights.
5. method according to claim 4, it is characterised in that:
It is described the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask artwork after fine segmentation, bag
Include:
Iteration performs following step C and D steps, stops iterative process after iteration operation reaches the condition of convergence, by the prospect
Each pixel in point set Q is used as the foreground point in the mask artwork after fine segmentation;
Wherein, step C and D steps include following treatment respectively:
Step C:One part of pixel in non-directed graph is divided into and vacantly point Q0Similar foreground point, by being divided into foreground point
Pixel constitutes prospect point set Q;
D steps:The weights summation of the prospect point set Q is calculated, before the weights summation is all in the prospect point set Q
The weights at sight spot and, along with all foreground points in the prospect point set Q and hanging point Q0Between line weights and;
Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability.
6. method according to claim 1, it is characterised in that:
Geometric figure of the construction comprising the smearing track simultaneously is extended to form mark zone to the geometric figure, it is determined that including
One piece of region of the smearing track as area-of-interest, including:
The minimum enclosed rectangle of the smearing track is constructed, the length L according to the smearing track is to the minimum enclosed rectangle
It is extended to form mark zone;
For geometry template is applied mechanically in the smearing track, the smearing track is included according to the geometry template generation
Area-of-interest, if the area-of-interest does not include the mark zone, the area-of-interest is extended to and is included
The mark zone;Or expansion process formation area-of-interest is carried out to the mark zone.
7. method according to claim 6, it is characterised in that:
The minimum enclosed rectangle of the construction smearing track, the length L according to the smearing track is external to the minimum
Rectangle is extended to form mark zone, including:
Transverse and longitudinal coordinate value according to all pixels on the smearing track determines maximum abscissa value x_max, minimum abscissa
Value x_min, maximum ordinate value y_max and minimum ordinate value y_min;
The minimum enclosed rectangle of the smearing track is constructed, the coordinate on four summits of the minimum enclosed rectangle is successively:(x_
Min, y_max), (x_min, y_min), (x_max, y_max), (x_max, y_min);
Length L according to the smearing track is extended to form mark zone to the minimum enclosed rectangle, the mark zone
Four apex coordinates are successively:(x_min-a, y_max+a), (x_min-a, y_min-a), (x_max+a, y_max+a), (x_
max+a,y_min-a);Wherein, a is adjustability coefficients, and a=L*b, b are constants.
8. method according to claim 1, it is characterised in that:
Before generating the input mask artwork of image segmentation algorithm, also include:
If currently detected smearing track is directed to the first time mark of the original image, by currently detected painting
The mark zone determined track is smeared as final mark zone, the area-of-interest that currently detected smearing track is determined
As final area-of-interest;
If currently detected smearing track is not directed to the first time mark of the original image, judge currently detected
The mark zone determined of smearing track and the mark zone determined of the smearing track that detects of last time between whether have common factor,
If occuring simultaneously, then the mark zone determined currently detected smearing track determines with the last smearing track for detecting
The mark zone for going out is merged, using the mark zone after merging as final mark zone;If do not occured simultaneously, by current detection
To the mark zone determined of smearing track as final mark zone;
If currently detected smearing track is not directed to the first time mark of the original image, judge currently detected
The area-of-interest determined of smearing track and the area-of-interest determined of the smearing track that detects of last time between be
It is no to have common factor, if occuring simultaneously, then the area-of-interest that currently detected smearing track is determined is detected with the last time
The area-of-interest determined of smearing track merge, using the area-of-interest after merging as final region of interest
Domain;If do not occured simultaneously, the area-of-interest that currently detected smearing track is determined is used as final region of interest
Domain.
9. a kind of device for realizing interactive image segmentation, including:
Pretreatment module, after detecting the smearing track on original image, geometric graph of the construction comprising the smearing track
Shape simultaneously is extended to form mark zone to the geometric figure, it is determined that one piece of region comprising the smearing track is used as interested
Region, the area-of-interest includes the mark zone;Generate the input mask artwork of image segmentation algorithm:By in the mark zone
All of pixel covers the pixel outside mark zone in the area-of-interest as described as the foreground point in the mask artwork
Background dot in mould figure;
Partitioning parameters computing module, for obtaining the depth map comprising destination object depth information, according to the depth map and covering
Mould figure determines the partitioning parameters of each pixel on the mask artwork, the partitioning parameters be used to representing pixel judgement for foreground point or
The depth value difference of the probability of background dot and the pixel and adjacent pixel;
Mask artwork adjusting module, for building non-directed graph, the partitioning parameters of each pixel in the mask artwork is mapped to
In the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask after fine segmentation
Figure;
Output module, for being partitioned into covering after the fine segmentation from the original image that the destination object is obtained is shot
The corresponding image in foreground point in mould figure.
10. the device for realizing interactive image segmentation described in a kind of terminal, including the claims 9.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710005330.3A CN106898005B (en) | 2017-01-04 | 2017-01-04 | Method, device and terminal for realizing interactive image segmentation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710005330.3A CN106898005B (en) | 2017-01-04 | 2017-01-04 | Method, device and terminal for realizing interactive image segmentation |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106898005A true CN106898005A (en) | 2017-06-27 |
CN106898005B CN106898005B (en) | 2020-07-17 |
Family
ID=59198345
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710005330.3A Active CN106898005B (en) | 2017-01-04 | 2017-01-04 | Method, device and terminal for realizing interactive image segmentation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106898005B (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107784658A (en) * | 2017-10-17 | 2018-03-09 | 南京信息工程大学 | The tactile texture display method of regular two-dimensional coloured image |
CN109447082A (en) * | 2018-08-31 | 2019-03-08 | 武汉尺子科技有限公司 | A kind of scene motion Target Segmentation method, system, storage medium and equipment |
CN110675940A (en) * | 2019-08-01 | 2020-01-10 | 平安科技(深圳)有限公司 | Pathological image labeling method and device, computer equipment and storage medium |
WO2020258793A1 (en) * | 2019-06-26 | 2020-12-30 | 北京市商汤科技开发有限公司 | Target detection and training of target detection network |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102903110A (en) * | 2012-09-29 | 2013-01-30 | 宁波大学 | Segmentation method for image with deep image information |
CN104463843A (en) * | 2014-10-31 | 2015-03-25 | 南京邮电大学 | Interactive image segmentation method of android system |
CN104820990A (en) * | 2015-05-15 | 2015-08-05 | 北京理工大学 | Interactive-type image-cutting system |
CN105787938A (en) * | 2016-02-25 | 2016-07-20 | 上海大学 | Figure segmentation method based on depth map |
US20160227208A1 (en) * | 2006-12-19 | 2016-08-04 | Playvision Technologies | Interactive imaging systems and methods for motion control by users |
-
2017
- 2017-01-04 CN CN201710005330.3A patent/CN106898005B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160227208A1 (en) * | 2006-12-19 | 2016-08-04 | Playvision Technologies | Interactive imaging systems and methods for motion control by users |
CN102903110A (en) * | 2012-09-29 | 2013-01-30 | 宁波大学 | Segmentation method for image with deep image information |
CN104463843A (en) * | 2014-10-31 | 2015-03-25 | 南京邮电大学 | Interactive image segmentation method of android system |
CN104820990A (en) * | 2015-05-15 | 2015-08-05 | 北京理工大学 | Interactive-type image-cutting system |
CN105787938A (en) * | 2016-02-25 | 2016-07-20 | 上海大学 | Figure segmentation method based on depth map |
Non-Patent Citations (1)
Title |
---|
胡卫平等: "一种数字图像区域处理技术的设计与应用", 《微计算机应用》 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107784658A (en) * | 2017-10-17 | 2018-03-09 | 南京信息工程大学 | The tactile texture display method of regular two-dimensional coloured image |
CN109447082A (en) * | 2018-08-31 | 2019-03-08 | 武汉尺子科技有限公司 | A kind of scene motion Target Segmentation method, system, storage medium and equipment |
CN109447082B (en) * | 2018-08-31 | 2020-09-15 | 武汉尺子科技有限公司 | Scene moving object segmentation method, system, storage medium and equipment |
WO2020258793A1 (en) * | 2019-06-26 | 2020-12-30 | 北京市商汤科技开发有限公司 | Target detection and training of target detection network |
CN110675940A (en) * | 2019-08-01 | 2020-01-10 | 平安科技(深圳)有限公司 | Pathological image labeling method and device, computer equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN106898005B (en) | 2020-07-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN104835165B (en) | Image processing method and image processing device | |
CN105354838B (en) | The depth information acquisition method and terminal of weak texture region in image | |
CN106846345A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106651867A (en) | Interactive image segmentation method and apparatus, and terminal | |
CN106898003A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106886999A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106898005A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106772247A (en) | A kind of terminal and sound localization method | |
CN106875399A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106534693A (en) | Photo processing method, photo processing device and terminal | |
CN106846323A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106708321A (en) | Touch screen touch method and device and terminal | |
CN106506778A (en) | A kind of dialing mechanism and method | |
CN106791111A (en) | A kind of images share method, device and terminal | |
CN106778887A (en) | The terminal and method of sentence flag sequence are determined based on condition random field | |
CN106570945A (en) | Terminal, check-in machine and check-in method | |
CN106780516A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106898002A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106875397A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106791119A (en) | A kind of photo processing method, device and terminal | |
CN106898004A (en) | A kind of preprocess method for realizing interactive image segmentation, device and terminal | |
CN106887009A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106651762A (en) | Photo processing method, device and terminal | |
CN106875398A (en) | A kind of method for realizing interactive image segmentation, device and terminal | |
CN106873981A (en) | A kind of icon processing method, device and terminal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |