CN106846323A

CN106846323A - A kind of method for realizing interactive image segmentation, device and terminal

Info

Publication number: CN106846323A
Application number: CN201710004573.5A
Authority: CN
Inventors: 梁舟
Original assignee: Nubia Technology Co Ltd
Current assignee: Zhuhai Dahengqin Technology Development Co Ltd
Priority date: 2017-01-04
Filing date: 2017-01-04
Publication date: 2017-06-13
Anticipated expiration: 2037-01-04
Also published as: CN106846323B

Abstract

Disclosed herein is a kind of method for realizing interactive image segmentation, device and terminal.Methods described includes：Sketching the contours after track carries out Seal treatment on original image is formed into mark zone, the input mask artwork of image segmentation algorithm is generated：Using the pixel in mark zone as the foreground point in mask artwork, using the pixel outside mark zone as the background dot in mask artwork；The first partitioning parameters of each pixel are determined according to cromogram and mask artwork, the second partitioning parameters of each pixel are determined according to depth map and mask artwork, two kinds of partitioning parameters are merged；Partitioning parameters after the fusion of each pixel are mapped in non-directed graph, mask artwork after treatment obtains fine segmentation is carried out to the non-directed graph according to minimal cut maximum-flow algorithm, the corresponding image in foreground point in the mask artwork after the fine segmentation is partitioned into from cromogram.The run time of algorithm can be shortened herein, the effect of image segmentation is improved with reference to the depth information of image.

Description

A kind of method for realizing interactive image segmentation, device and terminal

Technical field

The present invention relates to technical field of image processing, more particularly to a kind of method for realizing interactive image segmentation, device And terminal.

Background technology

Image segmentation, if refer to by plane picture according to the features such as color, texture, shape be divided into mutually it is disjunct Dry region, this is a basic technology for practicality of image processing field.Existing image Segmentation Technology has dividing based on threshold value Segmentation method, the dividing method based on edge, the dividing method based on region, the dividing method based on energy functional, based on graph theory Dividing method etc..Comparing wherein in Graph-theoretical Approach well-known has GraphCut algorithms and its modified version GrabCut algorithms.

GraphCut algorithms and its modified version GrabCut algorithms, are the interactive image segmentation sides based on area marking Method.GraphCut algorithms are based on Markov random field (Markov Random Field, MRF) energy minimization framework A kind of algorithm, advantage is that can carry out global optimum's solution with reference to various knowwhies.GrabCut algorithms are to GraphCut The improvement of algorithm, GrabCut algorithms by mark out on the original image foreground point (point on the destination object to be extracted) and Background dot generates mask artwork, and gauss hybrid models (Gaussian is set up to prospect, background color space using artwork and mask artwork Mixture Model, GMM), energy minimization is completed using the iterative algorithm that can be evolved in GMM parameter learnings, estimation procedure, The foreground point in image and background dot are ruled out, the target image being made up of foreground point pixel is extracted from artwork.

When carrying out image segmentation using GrabCut algorithms on mobile phone, in order to reduce interactive complexity, generally to user How to mark and be not strict with, therefore, in the case where the foreground point of user's mark is less, iterations may be a lot, algorithm Run time is more long, have impact on the experience of user.On the other hand, the GrabCut algorithms in correlation technique are based on coloured image Image segmentation is carried out, when the color characteristic of the destination object to be extracted is not obvious, using dividing that cromogram is split Cut effect unsatisfactory.

The content of the invention

The technical problems to be solved by the invention are to provide a kind of method for realizing interactive image segmentation, device and end End, can shorten the run time of algorithm, and the effect of image segmentation is improved with reference to the depth information of image.

A kind of method for realizing interactive image segmentation is the embodiment of the invention provides, including：

Detect on original image sketch the contours track after, described sketching the contours after track carries out Seal treatment is formed into mark zone, Generate the input mask artwork of image segmentation algorithm：Using all of pixel in the mark zone as the prospect in the mask artwork Point, using the pixel outside mark zone on the original image as the background dot in the mask artwork；

The cromogram comprising destination object colouring information and the depth map comprising destination object depth information are obtained, according to institute State cromogram and mask artwork determines the first partitioning parameters of each pixel on the mask artwork, according to the depth map and mask artwork Determine the second partitioning parameters of each pixel on the mask artwork, first partitioning parameters and the second partitioning parameters are used to represent Pixel judgement is the probability and the pixel of foreground point or background dot and the numerical value difference of adjacent pixel；By the described first segmentation Parameter is merged with second partitioning parameters；

Non-directed graph is built, the partitioning parameters after the fusion of each pixel in the mask artwork is mapped to described undirected In figure, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask artwork after fine segmentation；

The corresponding image in foreground point in the mask artwork after the fine segmentation is partitioned into from the cromogram.

Alternatively, the first partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork, Including：The first area partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork：

Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps；Iteration runs E Step and M steps, the iterative process is stopped after iteration operation reaches the condition of convergence；Last time is performed into M steps The classification of the pixel for obtaining is defined as the classification of the pixel, and the pixel that last time performs the acquisition of M steps is returned Belong to the most probable value P of certain cluster_maxIt is defined as the first area partitioning parameters of the pixel, the first area point It is the probability that the pixel is based on that cromogram judgement is foreground point or background dot to cut parameter；

Wherein, the E steps and M steps include following treatment respectively：

E steps：The position relationship between color value and pixel according to each pixel on the mask artwork is by clustering Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined；Wherein, the classification of pixel includes prospect Point or background dot；The classification of cluster includes foreground point cluster or background dot cluster；

M steps：GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one Pixel, the most probable value P according to the pixel_maxCorresponding cluster determines the classification of the pixel.

Alternatively, the second partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, Also include：The second area partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork：

Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps；Iteration runs E Step and M steps, the iterative process is stopped after iteration operation reaches the condition of convergence；Last time is performed into M steps The classification of the pixel for obtaining is defined as the classification of the pixel, and the pixel that last time performs the acquisition of M steps is returned Belong to the most probable value P of certain cluster_maxIt is defined as the second area partitioning parameters of the pixel, the second area point It is the probability that the pixel is based on that depth map judgement is foreground point or background dot to cut parameter；

Wherein, the E steps and M steps include following treatment respectively：

E steps：The position relationship between depth value and pixel according to each pixel on the mask artwork is by clustering Same type of pixel is polymerized to one or more clusters, the GMM model of each cluster is determined；Wherein, the classification of pixel includes prospect Point or background dot；The classification of cluster includes foreground point cluster or background dot cluster；

Alternatively, the first partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork, Also include：The first border item partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork：

Color distortion according to the pixel and adjacent pixel determines the first border item partitioning parameters of the pixel；

Wherein, to any one pixel, by tri- Color Channels of RGB between the pixel and each neighbouring pixel Quantity difference absolute value is added up, then to adding up and being normalized, the cumulative and conduct after being normalized is described First border item partitioning parameters of pixel.

Alternatively, the second partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, Also include：The second boundary partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork：

Depth value difference according to the pixel and adjacent pixel determines the second boundary partitioning parameters of the pixel；

Wherein, to any one pixel, by the depth value absolute difference between the pixel and each neighbouring pixel Added up, then to adding up and being normalized, adding up and as the second boundary of the pixel after being normalized Item partitioning parameters.

Alternatively, first partitioning parameters are merged with second partitioning parameters, including：By firstth area Domain partitioning parameters are merged with the second area partitioning parameters：

To any one pixel, the first area partitioning parameters are multiplied by after weight (1-a) is adjusted first The second area partitioning parameters are multiplied by a second area after weight a is adjusted segmentation ginseng by area item partitioning parameters Number；

If the pixel classifications that the first area partitioning parameters are indicated are indicated with the second area partitioning parameters Pixel classifications it is identical, then by the first area partitioning parameters after the adjustment and the second area segmentation after the adjustment Parameter be added and as merge after area item partitioning parameters；

If the pixel classifications that the first area partitioning parameters are indicated are indicated with the second area partitioning parameters Pixel classifications differ, then by the first area partitioning parameters after the adjustment and the second area point after the adjustment The pixel classifications of numerical value the greater instruction in parameter are cut as the final classification of the pixel, by the first area after the adjustment Item partitioning parameters are divided with the absolute value of the difference of the second area partitioning parameters after the adjustment as the area item after merging Cut parameter.

Alternatively, first partitioning parameters are merged with second partitioning parameters, is also included：By described first Border item partitioning parameters are merged with the second boundary partitioning parameters：

The first border item partitioning parameters are multiplied by the first border item partitioning parameters after weight (1-a) is adjusted, The second boundary partitioning parameters are multiplied by a second boundary after weight a is adjusted partitioning parameters, then by the tune The first border item partitioning parameters after whole are added with the second boundary partitioning parameters after the adjustment and obtain melting for the pixel Border item partitioning parameters after conjunction；A is more than or equal to 0 and less than or equal to 1.

Alternatively, the weight a is determined according to self-evaluation parameter k1 and parameter of consistency k2：By self-evaluation parameter k1 With the product of parameter of consistency k2 as weight a；

Wherein, self-evaluation parameter k1 is determined in the following ways：Depth value according to pixel determines that pixel is corresponding The how far of shooting distance, the how far according to the shooting distance sets self-evaluation parameter k1, and the shooting distance is got over Closely, self-evaluation parameter k1 sets bigger；K1 is more than or equal to 0 and less than or equal to 1；

Wherein, the parameter of consistency k2 is determined in the following ways：

If the first border item partitioning parameters are equal with the second boundary partitioning parameters, uniformity ginseng is set Number k2 is first constant；

If the first border item partitioning parameters are unequal with the second boundary partitioning parameters, on first side When boundary partitioning parameters are simultaneously greater than threshold value or are less than threshold value simultaneously with the second boundary partitioning parameters, uniformity ginseng is set Number k2 is first constant；In the first border item partitioning parameters threshold value is not simultaneously greater than with the second boundary partitioning parameters Or when asynchronously less than threshold value, it is second constant to set parameter of consistency k2；The first constant is more than the second constant；Institute First constant is stated more than 0 and less than or equal to 1, the second constant is more than 0 and less than 1.

Alternatively, build non-directed graph and be mapped to the partitioning parameters after the fusion of each pixel in the mask artwork In the non-directed graph, including：

A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph₀And Q₁, the hanging point Q₀For Virtual foreground point, the hanging point Q₁It is virtual background point；Each is set up on the mask artwork in the plane of the non-directed graph The mapping point of pixel, in mapping point and the hanging point Q of foreground point₀Between set up line, mapping point and institute in background dot State hanging point Q₁Between set up line；

To any one pixel P in mask artwork_i, by the pixel P_iFusion after area item partitioning parameters as institute State mapping point P in non-directed graph_i' weights, by the pixel P_iFusion after border item partitioning parameters as in the non-directed graph Mapping point P_i' with hanging point Q₀Or Q₁Between line weights.

Alternatively, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtains the mask after fine segmentation Figure, including：

Iteration performs following step C and D steps, and iterative process is stopped after iteration operation reaches the condition of convergence, will be described Each pixel in prospect point set Q is used as the foreground point in the mask artwork after fine segmentation；

Wherein, step C and D steps include following treatment respectively：

Step C：One part of pixel in non-directed graph is divided into and vacantly point Q₀Similar foreground point, by being divided into prospect The pixel of point constitutes prospect point set Q；

D steps：The weights summation of the prospect point set Q is calculated, the weights summation is institute in the prospect point set Q Have foreground point weights and, along with all foreground points in the prospect point set Q and hanging point Q₀Between line weights and；

Wherein, the condition of convergence be prospect point set Q weights summation less than threshold value and change tend towards stability.

Alternatively, it is described that described sketching the contours after track carries out Seal treatment is formed into mark zone, including：

It is described sketch the contours track it is closed when, the region closed track of sketching the contours is defined as mark zone；

It is described sketch the contours track it is not closed when, Seal treatment is done to the track of sketching the contours, if closed successfully, will closing The region closed track of sketching the contours afterwards is defined as mark zone, if closing is unsuccessful, the track of sketching the contours is expanded Treatment, mark zone is defined as by the region after the expansion.

Alternatively, it is described that Seal treatment is done to the track of sketching the contours, including：

If described the distance between beginning and end of track of sketching the contours is more than or equal to threshold value, obtain described in sketch the contours rail The edge line in region between the beginning and end of mark, the edge line is superimposed with by the track of sketching the contours；If the hook Le track is superimposed with the edge line can form closed area, then judge to close successfully, if the track of sketching the contours is superimposed with The edge line can not form closed area, then judge closing failure；

If described the distance between beginning and end of track of sketching the contours is less than threshold value, between the beginning and end Line segment connection is done, the closure of track is sketched the contours in completion.

Alternatively, described image partitioning algorithm is GrabCut algorithms.

A kind of device for realizing interactive image segmentation is the embodiment of the invention provides, including：

Pretreatment module, for detect on original image sketch the contours track after, the track of sketching the contours is carried out at closing Mark zone is formed after reason, the input mask artwork of image segmentation algorithm is generated：Using all of pixel in the mark zone as described Foreground point in mask artwork, using the pixel outside mark zone on the original image as the background dot in the mask artwork；

Partitioning parameters are calculated and Fusion Module, the cromogram of destination object colouring information are included for obtaining and comprising target The depth map of subject depth information, the first segmentation of each pixel on the mask artwork is determined according to the cromogram and mask artwork Parameter, the second partitioning parameters of each pixel on the mask artwork, described first point are determined according to the depth map and mask artwork Cut parameter and the second partitioning parameters be used to representing pixel judgement for foreground point or background dot probability and the pixel with it is adjacent The numerical value difference of pixel；First partitioning parameters are merged with second partitioning parameters；

Mask artwork adjusting module, for building non-directed graph, by the fusion of each pixel in the mask artwork after point Cut parameter to be mapped in the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain fine point Mask artwork after cutting；

Output module, it is corresponding for being partitioned into foreground point in the mask artwork after the fine segmentation from the cromogram Image.

Alternatively, partitioning parameters are calculated and Fusion Module, in the following ways according to the cromogram and mask artwork Determine the first partitioning parameters of each pixel on the mask artwork：Determined on the mask artwork according to the cromogram and mask artwork The first area partitioning parameters of each pixel：

Wherein, the E steps and M steps include following treatment respectively：

Alternatively, partitioning parameters are calculated and Fusion Module, are additionally operable in the following ways according to the depth map and mask Figure determines the second partitioning parameters of each pixel on the mask artwork：The mask artwork is determined according to the depth map and mask artwork The second area partitioning parameters of upper each pixel：

Wherein, the E steps and M steps include following treatment respectively：

Alternatively, partitioning parameters are calculated and Fusion Module, are additionally operable in the following ways according to the cromogram and mask Figure determines the first partitioning parameters of each pixel on the mask artwork：The mask artwork is determined according to the cromogram and mask artwork First border item partitioning parameters of upper each pixel：

Alternatively, partitioning parameters are calculated and Fusion Module, are additionally operable in the following ways according to the depth map and mask Figure determines the second partitioning parameters of each pixel on the mask artwork：The mask artwork is determined according to the depth map and mask artwork The second boundary partitioning parameters of upper each pixel：

Alternatively, partitioning parameters are calculated and Fusion Module, in the following ways by first partitioning parameters and institute The second partitioning parameters are stated to be merged：The first area partitioning parameters are melted with the second area partitioning parameters Close：

Alternatively, partitioning parameters calculate and Fusion Module, be additionally operable in the following ways will first partitioning parameters and Second partitioning parameters are merged：The first border item partitioning parameters are carried out with the second boundary partitioning parameters Fusion：

Wherein, the parameter of consistency k2 is determined in the following ways：

Alternatively, mask artwork adjusting module, for building non-directed graph and will be every in the mask artwork in the following ways Partitioning parameters after one fusion of pixel are mapped in the non-directed graph：

Alternatively, mask artwork adjusting module, in the following ways according to minimal cut-maximum-flow algorithm to described undirected Figure is processed, and obtains the mask artwork after fine segmentation：

Wherein, step C and D steps include following treatment respectively：

Alternatively, pretreatment module, for described sketching the contours after track carries out Seal treatment to be formed into mark in the following ways Note area, including：

Alternatively, pretreatment module, for doing Seal treatment to the track of sketching the contours in the following ways, including：

Alternatively, described image partitioning algorithm is GrabCut algorithms.

The embodiment of the present invention additionally provides a kind of terminal, including the above-mentioned device for realizing interactive image segmentation.

Set forth herein a kind of method for realizing interactive image segmentation, device and terminal, by sketching the contours on original image Track forms mark zone after carrying out Seal treatment, generates the input mask artwork of image segmentation algorithm：To own in the mark zone Pixel as the foreground point in the mask artwork, using the pixel outside mark zone on the original image as in the mask artwork Background dot；The cromogram comprising destination object colouring information and the depth map comprising destination object depth information are obtained, according to The cromogram and mask artwork determine the first partitioning parameters of each pixel on the mask artwork, according to the depth map and mask Figure determines the second partitioning parameters of each pixel on the mask artwork, and first partitioning parameters and the second partitioning parameters are used for table Show that pixel judgement is the probability and the pixel of foreground point or background dot and the numerical value difference of adjacent pixel；By described first point Parameter is cut to be merged with second partitioning parameters；Non-directed graph is built, by the fusion of each pixel in the mask artwork Partitioning parameters afterwards are mapped in the non-directed graph, and the non-directed graph is processed according to minimal cut-maximum-flow algorithm, are obtained Mask artwork after fine segmentation；The corresponding figure in foreground point in the mask artwork after the fine segmentation is partitioned into from the cromogram Picture.The prospect that the technical scheme of this paper can expand image segmentation algorithm mark by image preprocessing is counted out, so as to shorten The run time of image segmentation algorithm, the partitioning parameters for calculating each pixel respectively based on depth map and cromogram line parameter of going forward side by side melts Close, image segmentation is carried out using the partitioning parameters after fusion, image point is carried out compared to cromogram is simply used in correlation technique Cut, the effect of image segmentation can be improved.

Brief description of the drawings

Fig. 1 is the hardware architecture diagram for realizing each optional mobile terminal of embodiment one of the invention；

Fig. 2 is the wireless communication system schematic diagram of mobile terminal as shown in Figure 1；

Fig. 3 is a kind of method flow diagram for realizing interactive image segmentation of the embodiment of the present invention 1；

Fig. 4 is a kind of schematic device for realizing interactive image segmentation of the embodiment of the present invention 2；

Fig. 5-a in present invention application example 1 original image and user sketch the contours the schematic diagram of track (complete closure)；

Fig. 5-b are by sketching the contours the schematic diagram of the mark zone of Track Pick-up in present invention application example 1；

Fig. 5-c are the schematic diagram that the present invention applies the mask artwork generated by mark zone and original image in example 1；

Fig. 5-d are the schematic diagram of the depth map of present invention application example 1；

Fig. 5-e are the schematic diagram of the non-directed graph of present invention application example 1；

Fig. 5-f are the schematic diagram of the mask artwork after fine segmentation in present invention application example 1；

Fig. 5-g are the schematic diagram that the present invention applies the destination object being partitioned into example 1；

Fig. 6-a in present invention application example 2 original image and user sketch the contours track (non-close, breach be small, Ke Yitong Benefit line segment is crossed to be closed) schematic diagram；

Fig. 6-b are by sketching the contours the schematic diagram of the mark zone of Track Pick-up in present invention application example 2；

Fig. 6-c are the schematic diagram that the present invention applies the mask artwork generated by mark zone and original image in example 2；

Fig. 7-a-1 in present invention application example 3 original image and user sketch the contours track (non-close, breach be big, can be with Closed using profile) schematic diagram；

Fig. 7-a-2 in present invention application example 3 original image and user sketch the contours track (non-close, breach is big, it is impossible to Closed using profile) schematic diagram；

Fig. 7-b-1 are that by the mark zone that sketches the contours Track Pick-up, (non-close, breach is big, can be with profit in present invention application example 3 Closed with profile) schematic diagram；

Fig. 7-b-2 be the present invention application example 3 in by sketch the contours Track Pick-up mark zone (non-close, breach is big, it is impossible to profit Closed with profile) schematic diagram.

The realization of the object of the invention, functional characteristics and advantage will be described further referring to the drawings in conjunction with the embodiments.

Specific embodiment

Technical scheme is described in detail below in conjunction with drawings and Examples.

The mobile terminal of each embodiment of the application is realized referring now to Description of Drawings.In follow-up description, use For represent element such as " module ", " part " or " unit " suffix only for being conducive to explanation of the invention, itself Not specific meaning.Therefore, " module " can be used mixedly with " part ".

Mobile terminal can be implemented in a variety of manners.For example, the terminal described in the present invention can include such as moving Phone, smart phone, notebook computer, digit broadcasting receiver, PDA (personal digital assistant), PAD (panel computer), PMP The mobile terminal of (portable media player), guider etc. and such as numeral TV, desktop computer etc. are consolidated Determine terminal.Hereinafter it is assumed that terminal is mobile terminal.However, it will be understood by those skilled in the art that, except being used in particular for movement Outside the element of purpose, construction according to the embodiment of the present invention can also apply to the terminal of fixed type.

Fig. 1 is to realize the application one hardware architecture diagram of optional mobile terminal of each embodiment.

Mobile terminal 1 00 can include wireless communication unit 110, A/V (audio/video) input block 120, user input Unit 130, sensing unit 140, output unit 150, memory 160, interface unit 170, controller 180 and power subsystem 190 Etc..

Fig. 1 shows the mobile terminal 1 00 with various assemblies, it should be understood that being not required for implementing all showing The component for going out.More or less component can alternatively be implemented.The element of mobile terminal 1 00 will be discussed in more detail below.

Wireless communication unit 110 can generally include one or more assemblies, and it allows mobile terminal 1 00 and radio communication Radio communication between system or network.For example, wireless communication unit 110 can include that broadcasting reception module 111, movement are logical At least one of letter module 112, wireless Internet module 113, short range communication module 114 and location information module 115.

Broadcasting reception module 111 receives broadcast singal and/or broadcast via broadcast channel from external broadcast management server Relevant information.Broadcast channel can include satellite channel and/or terrestrial channel.Broadcast management server can be generated and sent The broadcast singal and/or broadcast related information generated before the server or reception of broadcast singal and/or broadcast related information And send it to the server of terminal.Broadcast singal can include TV broadcast singals, radio signals, data broadcasting Signal etc..And, broadcast singal may further include the broadcast singal combined with TV or radio signals.Broadcast phase Pass information can also be provided via mobile communications network, and in said case, broadcast related information can be by mobile communication Module 112 is received.Broadcast singal can exist in a variety of manners, for example, it can be with the electricity of DMB (DMB) The form of sub- program guide (EPG), the electronic service guidebooks (ESG) of digital video broadcast-handheld (DVB-H) etc. and exist.Extensively Broadcasting receiver module 111 can receive signal broadcast by using various types of broadcast systems.Especially, broadcasting reception module 111 can be by using such as multimedia broadcasting-ground (DMB-T), DMB-satellite (DMB-S), digital video Broadcasting-Handheld (DVB-H), Radio Data System, the received terrestrial digital broadcasting integrated service of forward link media (MediaFLO@) Etc. (ISDB-T) digit broadcasting system receives digital broadcasting.Broadcasting reception module 111 may be constructed such that and be adapted to provide for extensively Broadcast the various broadcast systems and above-mentioned digit broadcasting system of signal.Via broadcasting reception module 111 receive broadcast singal and/ Or broadcast related information can be stored in memory 160 (or other types of storage medium).

Mobile communication module 112 sends radio signals to base station (for example, access point, node B etc.), exterior terminal And at least one of server and/or receive from it radio signal.Such radio signal can be logical including voice Words signal, video calling signal or the various types of data for sending and/or receiving according to text and/or Multimedia Message.

Wireless Internet module 113 supports the Wi-Fi (Wireless Internet Access) of mobile terminal.The module can be with internal or external Be couple to terminal.Wi-Fi (Wireless Internet Access) technology involved by the module can include WLAN (WLAN) (Wi-Fi), Wibro (WiMAX), Wimax (worldwide interoperability for microwave accesses), HSDPA (high-speed downlink packet access) etc..

Short range communication module 114 is the module for supporting junction service.Some examples of short-range communication technology include indigo plant Tooth TM, radio frequency identification (RFID), Infrared Data Association (IrDA), ultra wide band (UWB), purple honeybee TM etc..

Location information module 115 is the module for checking or obtaining the positional information of mobile terminal.Location information module 115 typical case is GPS (global positioning system).According to current technology, GPS calculate from three or more satellites away from Information application triangulation from information and correct time information and for calculating, so as to according to longitude, latitude and height Degree calculates three-dimensional current location information exactly.Currently, three satellites are used simultaneously for calculating the method for position and temporal information And the error of the position and temporal information for calculating is corrected by using an other satellite.Additionally, GPS can be by real-time Ground Continuous plus current location information carrys out calculating speed information.

A/V input blocks 120 are used to receive audio or video signal.A/V input blocks 120 can include the He of camera 121 Microphone 122, the static images that 121 pairs, camera is obtained in Video Capture pattern or image capture mode by image capture apparatus Or the view data of video is processed.Picture frame after treatment may be displayed on display unit 151.Processed through camera 121 Picture frame afterwards can be stored in memory 160 (or other storage mediums) or sent out via wireless communication unit 110 Send, two or more cameras 121 can be provided according to the construction of mobile terminal 1 00.Microphone 122 can be in telephone relation mould In formula, logging mode, speech recognition mode etc. operational mode sound (voice data), and energy are received via microphone 122 Enough is voice data by such acoustic processing.Audio (voice) data after treatment can be in the case of telephone calling model The form that being converted to can be sent to mobile communication base station via mobile communication module 112 is exported.Microphone 122 can be implemented various The noise of type eliminates (or suppression) algorithm and is being received and making an uproar of producing during sending audio signal with eliminating (or suppression) Sound or interference.

User input unit 130 can generate key input data to control mobile terminal 1 00 according to the order of user input Various operations.User input unit 130 allow the various types of information of user input, and can include keyboard, metal dome, Touch pad (for example, detection due to being touched caused by resistance, pressure, electric capacity etc. change sensitive component), roller, shake Bar etc..Especially, when touch pad is superimposed upon on display unit 151 in the form of layer, touch-screen can be formed.

Sensing unit 140 detects the current state of mobile terminal 1 00, (for example, mobile terminal 1 00 opens or closes shape State), the presence or absence of the contact (that is, touch input) of the position of mobile terminal 1 00, user for mobile terminal 1 00, mobile terminal The acceleration or deceleration movement of 100 orientation, mobile terminal 1 00 and direction etc., and generate for controlling mobile terminal 1 00 The order of operation or signal.For example, when mobile terminal 1 00 is embodied as sliding-type mobile phone, sensing unit 140 can be sensed The sliding-type phone is opened or closed.In addition, sensing unit 140 can detect whether power subsystem 190 provides electric power Or whether interface unit 170 couples with external device (ED).Sensing unit 140 can include proximity transducer 141.

Interface unit 170 is connected the interface that can pass through with mobile terminal 1 00 as at least one external device (ED).For example, External device (ED) can include wired or wireless head-band earphone port, external power source (or battery charger) port, wired or nothing Line FPDP, memory card port, the port for connecting the device with identification module, audio input/output (I/O) end Mouth, video i/o port, ear port etc..Identification module can be that storage uses each of mobile terminal 1 00 for verifying user Kind of information and subscriber identification module (UIM), client identification module (SIM), Universal Subscriber identification module (USIM) can be included Etc..In addition, the device (hereinafter referred to as " identifying device ") with identification module can take the form of smart card, therefore, know Other device can be connected via port or other attachment means with mobile terminal 1 00.Interface unit 170 can be used for reception and come from The input (for example, data message, electric power etc.) of the external device (ED) and input that will be received is transferred in mobile terminal 1 00 One or more elements can be used for transmitting data between mobile terminal 1 00 and external device (ED).

In addition, when mobile terminal 1 00 is connected with external base, interface unit 170 can serve as allowing by it by electricity Power provides to the path of mobile terminal 1 00 from base or can serve as allowing the various command signals being input into from base to pass through it It is transferred to the path of mobile terminal 1 00.Can serve as recognizing mobile terminal 1 00 from the various command signals or electric power of base input Whether signal base on is accurately fitted within.Output unit 150 is configured to be provided with vision, audio and/or tactile manner Output signal (for example, audio signal, vision signal, alarm signal, vibration signal etc.).Output unit 150 can include aobvious Show unit 151, dio Output Modules 152, alarm unit 153 etc..

Display unit 151 may be displayed on the information processed in mobile terminal 1 00.For example, when mobile terminal 1 00 is in electricity During words call mode, display unit 151 can show and converse or other communicate (for example, text messaging, multimedia file Download etc.) related user interface (UI) or graphic user interface (GUI).When mobile terminal 1 00 is in video calling pattern Or during image capture mode, display unit 151 can show the image of capture and/or the image of reception, show video or figure UI or GUI of picture and correlation function etc..

Meanwhile, when display unit 151 and touch pad in the form of layer it is superposed on one another to form touch-screen when, display unit 151 can serve as input unit and output device.Display unit 151 can include liquid crystal display (LCD), thin film transistor (TFT) In LCD (TFT-LCD), Organic Light Emitting Diode (OLED) display, flexible display, three-dimensional (3D) display etc. at least It is a kind of.Some in these displays may be constructed such that transparence to allow user to be watched from outside, and this is properly termed as transparent Display, typical transparent display can be, for example, TOLED (transparent organic light emitting diode) display etc..According to specific Desired implementation method, mobile terminal 1 00 can include two or more display units (or other display devices), for example, moving Dynamic terminal 100 can include outernal display unit (not shown) and inner display unit (not shown).Touch-screen can be used to detect Touch input pressure and touch input position and touch input area.

Dio Output Modules 152 can be in call signal reception pattern, call mode, record mould in mobile terminal 1 00 It is that wireless communication unit 110 is received or in memory when under the isotypes such as formula, speech recognition mode, broadcast reception mode In 160 store voice data transducing audio signal and be output as sound.And, dio Output Modules 152 can provide with The audio output of the specific function correlation that mobile terminal 1 00 is performed is (for example, call signal receives sound, message sink sound etc. Deng).Dio Output Modules 152 can include loudspeaker, buzzer etc..

Alarm unit 153 can provide output and be notified to mobile terminal 1 00 with by event.Typical event can be with Including calling reception, message sink, key signals input, touch input etc..In addition to audio or video is exported, alarm unit 153 can in a different manner provide output with the generation of notification event.For example, alarm unit 153 can be in the form of vibrating Output is provided, when calling, message or some other entrance communication (incoming communication) are received, alarm list Unit 153 can provide tactile output (that is, vibrating) to notify to user.Exported by providing such tactile, even if When in pocket of the mobile phone of user in user, user also can recognize that the generation of various events.Alarm unit 153 The output of the generation of notification event can be provided via display unit 151 or dio Output Modules 152.

Memory 160 can store software program for the treatment and control operation performed by controller 180 etc., Huo Zheke Temporarily to store oneself data (for example, telephone directory, message, still image, video etc.) through exporting or will export.And And, memory 160 can store the vibration of various modes on being exported when touching and being applied to touch-screen and audio signal Data.

Memory 160 can include the storage medium of at least one type, and the storage medium includes flash memory, hard disk, many Media card, card-type memory (for example, SD or DX memories etc.), random access storage device (RAM), static random-access storage Device (SRAM), read-only storage (ROM), Electrically Erasable Read Only Memory (EEPROM), programmable read only memory (PROM), magnetic storage, disk, CD etc..And, mobile terminal 1 00 can perform memory with by network connection The network storage device cooperation of 160 store function.

The overall operation of the generally control mobile terminal of controller 180.For example, controller 180 is performed and voice call, data Communication, video calling etc. related control and treatment.In addition, controller 180 can be included for reproducing (or playback) many matchmakers The multi-media module 181 of volume data, multi-media module 181 can be constructed in controller 180, or can be structured as and control Device 180 is separated.Controller 180 can be with execution pattern identifying processing, the handwriting input that will be performed on the touchscreen or picture Draw input and be identified as character or image.

Power subsystem 190 receives external power or internal power under the control of controller 180 and provides operation each unit Appropriate electric power needed for part and component.

Various implementation methods described herein can be with use such as computer software, hardware or its any combination of calculating Machine computer-readable recording medium is implemented.Implement for hardware, implementation method described herein can be by using application-specific IC (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), scene can Programming gate array (FPGA), processor, controller, microcontroller, microprocessor, it is designed to perform function described herein At least one in electronic unit is implemented, and in some cases, such implementation method can be implemented in controller 180. For software implementation, the implementation method of such as process or function can with allow to perform the single of at least one function or operation Software module is implemented.Software code can be come by the software application (or program) write with any appropriate programming language Implement, software code can be stored in memory 160 and performed by controller 180.

So far, oneself according to its function through describing mobile terminal 1 00.In addition, the mobile terminal 1 00 in the embodiment of the present invention Can be such as folded form, board-type, oscillating-type, sliding-type and other various types of mobile terminals, not do herein specifically Limit.

Mobile terminal 1 00 as shown in Figure 1 may be constructed such that using via frame or packet transmission data it is all if any Line and wireless communication system and satellite-based communication system are operated.

The communication system that mobile terminal wherein of the invention can be operated is described referring now to Fig. 2.

Such communication system can use different air interface and/or physical layer.For example, used by communication system Air interface includes such as frequency division multiple access (FDMA), time division multiple acess (TDMA), CDMA (CDMA) and universal mobile communications system System (UMTS) (especially, Long Term Evolution (LTE)), global system for mobile communications (GSM) etc..As non-limiting example, under The description in face is related to cdma communication system, but such teaching is equally applicable to other types of system.

With reference to Fig. 2, cdma wireless communication system can include multiple mobile terminal 1s 00, multiple base station (BS) 270, base station Controller (BSC) 275 and mobile switching centre (MSC) 280.MSC 280 is configured to and Public Switched Telephony Network (PSTN) 290 form interface.MSC 280 is also structured to be formed with the BSC 275 that can be couple to base station 270 via back haul link and connects Mouthful.If any one in the interface that back haul link can be known according to Ganji is constructed, the interface can include such as Europe mark Quasi- high power capacity digital circuit/Unite States Standard high power capacity digital circuit (E1/T1), asynchronous transfer mode (ATM), procotol (IP), point-to-point protocol (PPP), frame relay, high-bit-rate digital subscriber line road (HDSL), Asymmetrical Digital Subscriber Line (ADSL) Or all kinds digital subscriber line (xDSL).It will be appreciated that system can include multiple BSC 275 as shown in Figure 2.

Each BS 270 can service one or more subregions (or region), by multidirectional antenna or the day of sensing specific direction Each subregion of line covering is radially away from BS 270.Or, each subregion can by two for diversity reception or more Multiple antennas are covered.Each BS 270 may be constructed such that the multiple frequency distribution of support, and the distribution of each frequency has specific frequency Spectrum (for example, 1.25MHz, 5MHz etc.).

What subregion and frequency were distributed intersects can be referred to as CDMA Channel.BS 270 can also be referred to as base station transceiver System (BTS) or other equivalent terms.In this case, term " base station " can be used for broadly representing single BSC 275 and at least one BS 270.Base station can also be referred to as " cellular station ".Or, each subregion of specific BS 270 can be claimed It is multiple cellular stations.

As shown in Figure 2, broadcast singal is sent to broadcsting transmitter (BT) 295 mobile terminal operated in system 100.Broadcasting reception module 111 as shown in Figure 1 is arranged at mobile terminal 1 00 to receive the broadcast sent by BT 295 Signal.In fig. 2 it is shown that several global positioning system (GPS) satellites 300.Satellite 300 helps position multiple mobile terminals At least one of 100.

In fig. 2, multiple satellites 300 are depicted, it is understood that be, it is possible to use any number of satellite obtains useful Location information.Location information module 115 as shown in Figure 1 is (such as：GPS) it is generally configured to coordinate to obtain with satellite 300 The location information that must be wanted.Substitute GPS tracking techniques or outside GPS tracking techniques, it is possible to use can track mobile whole Other technologies of the position at end.In addition, at least one gps satellite 300 can optionally or additionally process satellite dmb biography It is defeated.

Used as a typical operation of wireless communication system, BS 270 receives the reverse strand from various mobile terminal 1s 00 Road signal.Mobile terminal 1 00 generally participates in call, information receiving and transmitting and other types of communication.Each of certain base station reception is anti- Processed in specific BS 270 to link signal.The data of acquisition are forwarded to the BSC 275 of correlation.BSC provides logical Words resource allocation and the mobile management function of the coordination including the soft switching process between BS 270.BSC 275 will also be received Data be routed to MSC 280, its provide for PSTN 290 formed interface extra route service.Similarly, PSTN 290 form interface with MSC 280, and MSC and BSC 275 form interface, and BSC 275 correspondingly controls BS 270 with by forward direction Link signal is sent to mobile terminal 1 00.

Based on above-mentioned mobile terminal hardware configuration and communication system, the application method each embodiment is proposed.

As shown in figure 3, the embodiment of the present invention proposes a kind of method for realizing interactive image segmentation, including：

S310, detect on original image sketch the contours track after, the track of sketching the contours is carried out forming mark after Seal treatment Note area, generates the input mask artwork of image segmentation algorithm：Using all of pixel in the mark zone as in the mask artwork Foreground point, using the pixel outside mark zone on the original image as the background dot in the mask artwork；

S320, obtains the cromogram comprising destination object colouring information and the depth map comprising destination object depth information, The first partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork, according to the depth map and Mask artwork determines the second partitioning parameters of each pixel on the mask artwork, and first partitioning parameters and the second partitioning parameters are used In the probability and the pixel and the numerical value difference of adjacent pixel that represent that pixel judgement is foreground point or background dot；By described One partitioning parameters are merged with second partitioning parameters；

S330, builds non-directed graph, and the partitioning parameters after the fusion of each pixel in the mask artwork are mapped into institute State in non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask artwork after fine segmentation；

S340, is partitioned into the corresponding image in foreground point in the mask artwork after the fine segmentation from the cromogram；

Methods described can also include following features：

Wherein, it is a kind of mode of mark destination object to sketch the contours, and it is to enter rower along the exterior contour of destination object to sketch the contours Note；

Wherein, mask artwork refers to that the part or all of pixel of a sub-picture has been carried out to be generated after prospect is distinguished with background Mark figure, each pixel on the mask artwork is marked as foreground point or background dot；

In the present embodiment, described image partitioning algorithm is GrabCut algorithms.

In the present embodiment, described sketching the contours after track carries out Seal treatment is formed into mark zone, including：

Wherein, Seal treatment is done to the track of sketching the contours, including：

In the present embodiment, the first segmentation of each pixel on the mask artwork is determined according to the cromogram and mask artwork Parameter, including：The first area partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork：

Wherein, the E steps and M steps include following treatment respectively：

M steps：GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one Pixel, the most probable value P according to the pixel_maxCorresponding cluster determines the classification of the pixel；

In the present embodiment, the second segmentation of each pixel on the mask artwork is determined according to the depth map and mask artwork Parameter, also includes：The second area segmentation ginseng of each pixel on the mask artwork is determined according to the depth map and mask artwork Number：

Wherein, the E steps and M steps include following treatment respectively：

Alternatively, the condition of convergence of the EM methods can be：Stop iteration when the number of times of iteration operation reaches threshold value Process；

In the present embodiment, the first segmentation of each pixel on the mask artwork is determined according to the cromogram and mask artwork Parameter, also includes：The first border item for determining each pixel on the mask artwork according to the cromogram and mask artwork splits ginseng Number：

Wherein, to any one pixel, by tri- Color Channels of RGB between the pixel and each neighbouring pixel Quantity difference absolute value is added up, then to adding up and being normalized, the cumulative and conduct after being normalized is described First border item partitioning parameters of pixel；

Wherein, the span of the numerical value on tri- Color Channels of RGB is：0~255；

In the present embodiment, the second segmentation of each pixel on the mask artwork is determined according to the depth map and mask artwork Parameter, also includes：The second boundary segmentation ginseng of each pixel on the mask artwork is determined according to the depth map and mask artwork Number：

Wherein, to any one pixel, by the depth value absolute difference between the pixel and each neighbouring pixel Added up, then to adding up and being normalized, adding up and as the second boundary of the pixel after being normalized Item partitioning parameters；

Wherein, the number range of depth value can be：0~255；

Alternatively, each pixel neighbouring with pixel can be 8 pixels on the pixel periphery.

In the present embodiment, first partitioning parameters are merged with second partitioning parameters, including：Will be described First area partitioning parameters are merged with the second area partitioning parameters：

If the pixel classifications that the first area partitioning parameters are indicated are indicated with the second area partitioning parameters Pixel classifications differ, then by the first area partitioning parameters after the adjustment and the second area point after the adjustment The pixel classifications of numerical value the greater instruction in parameter are cut as the final classification of the pixel, by the first area after the adjustment Item partitioning parameters are divided with the absolute value of the difference of the second area partitioning parameters after the adjustment as the area item after merging Cut parameter；

In the present embodiment, first partitioning parameters are merged with second partitioning parameters, is also included：By institute The first border item partitioning parameters are stated to be merged with the second boundary partitioning parameters：

The first border item partitioning parameters are multiplied by the first border item partitioning parameters after weight (1-a) is adjusted, The second boundary partitioning parameters are multiplied by a second boundary after weight a is adjusted partitioning parameters, then by the tune The first border item partitioning parameters after whole are added with the second boundary partitioning parameters after the adjustment and obtain melting for the pixel Border item partitioning parameters after conjunction；A is more than or equal to 0 and less than or equal to 1；

In the present embodiment, the weight a is determined according to self-evaluation parameter k1 and parameter of consistency k2：By self-evaluation The product of parameter k1 and parameter of consistency k2 is used as weight a；

Wherein, the parameter of consistency k2 is determined in the following ways：

If the first border item partitioning parameters are unequal with the second boundary partitioning parameters, on first side When boundary partitioning parameters are simultaneously greater than threshold value or are less than threshold value simultaneously with the second boundary partitioning parameters, uniformity ginseng is set Number k2 is first constant；In the first border item partitioning parameters threshold value is not simultaneously greater than with the second boundary partitioning parameters Or when asynchronously less than threshold value, it is second constant to set parameter of consistency k2；The first constant is more than the second constant；Institute First constant is stated more than 0 and less than or equal to 1, the second constant is more than 0 and less than 1；

In the present embodiment, non-directed graph is built and by the partitioning parameters after the fusion of each pixel in the mask artwork It is mapped in the non-directed graph, including：

In the present embodiment, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, after obtaining fine segmentation Mask artwork, including：

Wherein, step C and D steps include following treatment respectively：

In correlation technique, the foreground point in the input mask artwork of image segmentation algorithm is user's hand labeled, before existing Sight spot mark is less to cause image segmentation algorithm to be to distinguish foreground point and the iterations of background dot to increase, Riming time of algorithm Problem long.After using the method for the embodiment of the present invention, mark zone is used as in the region closed by sketching the contours track, in mark zone All pixels are labeled as foreground point, so that the prospect for being marked in the input mask artwork of expansion image segmentation algorithm automatically is counted out, It is the iterations for distinguishing foreground point and background dot that image segmentation algorithm can be reduced, and significantly decreases the fortune of image segmentation algorithm The row time.On the other hand, the technical scheme of the embodiment of the present invention can respectively calculate each pixel based on depth map and cromogram Partitioning parameters go forward side by side line parameter fusion, carry out image segmentation using the partitioning parameters after fusion, compared in correlation technique only Image segmentation is carried out using cromogram, the effect of image segmentation can be improved.

As shown in figure 4, the embodiment of the present invention proposes a kind of device for realizing interactive image segmentation, including：

Pretreatment module 401, for detect on original image sketch the contours track after, the track of sketching the contours is closed Mark zone is formed after treatment, the input mask artwork of image segmentation algorithm is generated：Using all of pixel in the mark zone as institute The foreground point in mask artwork is stated, using the pixel outside mark zone on the original image as the background dot in the mask artwork；

Partitioning parameters calculate and Fusion Module 402, for obtain comprising destination object colouring information cromogram and comprising The depth map of destination object depth information, first of each pixel on the mask artwork is determined according to the cromogram and mask artwork Partitioning parameters, the second partitioning parameters of each pixel on the mask artwork, described are determined according to the depth map and mask artwork One partitioning parameters and the second partitioning parameters be used to representing pixel judgement for foreground point or background dot probability and the pixel with The numerical value difference of adjacent pixel；First partitioning parameters are merged with second partitioning parameters；

Mask artwork adjusting module 403, for building non-directed graph, after the fusion of each pixel in the mask artwork Partitioning parameters are mapped in the non-directed graph, and the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain fine Mask artwork after segmentation；

Output module 404, for being partitioned into foreground point pair in the mask artwork after the fine segmentation from the cromogram The image answered；

Described device can also include following features：

In one embodiment, pretreatment module, in the following ways carrying out at closing the track of sketching the contours Mark zone is formed after reason, including：

Wherein, pretreatment module, for doing Seal treatment to the track of sketching the contours in the following ways, including：

In the present embodiment, partitioning parameters are calculated and Fusion Module, in the following ways according to the cromogram and Mask artwork determines the first partitioning parameters of each pixel on the mask artwork：Covered according to the cromogram and mask artwork determine The first area partitioning parameters of each pixel on mould figure：

Wherein, the E steps and M steps include following treatment respectively：

In the present embodiment, partitioning parameters are calculated and Fusion Module, are additionally operable in the following ways according to the depth map With the second partitioning parameters that mask artwork determines each pixel on the mask artwork：According to the depth map and mask artwork determine The second area partitioning parameters of each pixel on mask artwork：

Wherein, the E steps and M steps include following treatment respectively：

In the present embodiment, partitioning parameters are calculated and Fusion Module, are additionally operable in the following ways according to the cromogram With the first partitioning parameters that mask artwork determines each pixel on the mask artwork：According to the cromogram and mask artwork determine First border item partitioning parameters of each pixel on mask artwork：

In the present embodiment, partitioning parameters are calculated and Fusion Module, are additionally operable in the following ways according to the depth map With the second partitioning parameters that mask artwork determines each pixel on the mask artwork：According to the depth map and mask artwork determine The second boundary partitioning parameters of each pixel on mask artwork：

Wherein, the number range of depth value can be：0~255；

In the present embodiment, partitioning parameters are calculated and Fusion Module, in the following ways joining the described first segmentation Number is merged with second partitioning parameters：By the first area partitioning parameters and the second area partitioning parameters Merged：

In the present embodiment, partitioning parameters are calculated and Fusion Module, are additionally operable to the described first segmentation in the following ways Parameter is merged with second partitioning parameters：By the first border item partitioning parameters and the second boundary segmentation ginseng Number is merged：

Wherein, the parameter of consistency k2 is determined in the following ways：

In the present embodiment, mask artwork adjusting module, for building non-directed graph in the following ways and by the mask artwork In each pixel fusion after partitioning parameters be mapped in the non-directed graph：

Wherein, mask artwork adjusting module, in the following ways according to minimal cut-maximum-flow algorithm to the non-directed graph Processed, obtained the mask artwork after fine segmentation：

Wherein, step C and D steps include following treatment respectively：

The method of the embodiment of the present invention, it is all in mark zone by the way that the region closed track as mark zone will be sketched the contours Pixel is labeled as foreground point, so that the prospect for being marked in the input mask artwork of expansion image segmentation algorithm automatically is counted out, can subtract Few image segmentation algorithm is the iterations for distinguishing foreground point and background dot, when significantly decreasing the operation of image segmentation algorithm Between.On the other hand, the technical scheme of the embodiment of the present invention can be based on depth map and cromogram calculates the segmentation of each pixel respectively Parameter is gone forward side by side line parameter fusion, image segmentation is carried out using the partitioning parameters after fusion, compared to being simply used in correlation technique Cromogram carries out image segmentation, can improve the effect of image segmentation.

Embodiment 3

The embodiment of the present invention also provides a kind of terminal, and the terminal includes the above-mentioned device for realizing interactive image segmentation.

Using example 1

User's destination object interested in oneself on the original image is sketched the contours, using the image segmentation side of this paper Method is extracted to the destination object, be may comprise steps of：

Step S501, detects user's selection destination object is marked by the way of sketching the contours；

Such as, two buttons for being used to mark are provided on interface, one is " smearing ", and one is " sketching the contours ", if user " sketching the contours " button is clicked, has then been pre-processed to sketching the contours track.

Wherein, it is two kinds of different modes for marking destination object to smear and sketch the contours；

Usually, smearing is marked in the interior zone of destination object, and it is along the outer wheels of destination object to sketch the contours Exterior feature is marked；

Step S502, detects user and is sketched the contours on the original image；

Such as, as shown in Fig. 5-a, user is sketched the contours on the original image, and destination object is " stapler ", the hook It is closed curve to strangle track.

Step S503, mark zone is formed by described sketching the contours after track carries out Seal treatment；

Wherein, it is described sketch the contours track it is closed when, the region closed track of sketching the contours is defined as mark zone；

Such as, as shown in Fig. 5-b, the track of sketching the contours is closed curve, then sketch the contours the region that track is closed by described It is defined as mark zone；

Step S504, the input mask artwork of generation image segmentation algorithm (GrabCut algorithms)：To own in the mark zone Pixel as the foreground point in mask artwork, using the pixel outside mark zone on the original image as the background in mask artwork Point.

Such as, as shown in Fig. 5-c, the irregular darker regions for sketching the contours track closing are mark zones, and the mark zone is to cover Foreground point block in mould figure (input mask artwork)；The edge of original image is indicated with dotted line frame, is removed in the dotted line frame The part for going to mark zone (foreground point block) is the background dot block in mask artwork.

Step S505, obtains cromogram, and the depth map comprising destination object depth information；

As shown in Fig. 5-a, the original image that user is smeared is cromogram；

As shown in Fig. 5-d, depth map is a figure comprising depth information, and the size with cromogram is consistent；Depth In figure, farther out, the shallower part shooting distance of color is nearer for the deeper part shooting distance of color.

Step S506, the first segmentation ginseng of each pixel on the mask artwork is determined according to the cromogram and mask artwork Number, the second partitioning parameters of each pixel on the mask artwork, first segmentation are determined according to the depth map and mask artwork Parameter and the second partitioning parameters are used to represent that pixel judgement is foreground point or the probability and the pixel and adjacent picture of background dot The numerical value difference of element；First partitioning parameters are merged with second partitioning parameters；

Wherein, to any one pixel on the mask artwork, the partitioning parameters of the pixel include area item partitioning parameters With border item partitioning parameters；It is general for foreground point or background dot that the area item partitioning parameters of the pixel refer to pixel judgement Rate；The border item partitioning parameters of the pixel refer to the numerical value difference of the pixel and adjacent pixel；

Wherein, the first area segmentation ginseng of each pixel on the mask artwork is determined according to the cromogram and mask artwork Number：

Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps；Iteration runs E Step and M steps, the iterative process is stopped after iteration operation reaches predetermined number of times herein；Last time is performed into M The classification of the pixel that step is obtained is defined as the classification of the pixel, and last time is performed into the picture that M steps are obtained Element belongs to the most probable value P of certain cluster_maxIt is defined as the first area partitioning parameters of the pixel, the first area Item partitioning parameters are the probability that the pixel is based on that cromogram judgement is foreground point or background dot；

Wherein, the E steps and M steps include following treatment respectively：

Wherein, the color value of each pixel can be rgb value；

Wherein, the second area segmentation ginseng of each pixel on the mask artwork is determined according to the depth map and mask artwork Number：

Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps；Iteration runs E Step and M steps, the iterative process is stopped after the iteration number of run reaches predetermined number of times；Last time is performed into M The classification of the pixel that step is obtained is defined as the classification of the pixel, and last time is performed into the picture that M steps are obtained Element belongs to the most probable value P of certain cluster_maxIt is defined as the second area partitioning parameters of the pixel, the second area Item partitioning parameters are the probability that the pixel is based on that depth map judgement is foreground point or background dot；

Wherein, the E steps and M steps include following treatment respectively：

Wherein, the color distortion according to the pixel and adjacent pixel determines that the first border item of the pixel splits ginseng Number, including：To any one pixel, by the numerical value on tri- Color Channels of RGB between the pixel and each neighbouring pixel Absolute difference is added up, then to adding up and being normalized, adding up and as the pixel after being normalized The first border item partitioning parameters；

Wherein, the depth value difference according to the pixel and adjacent pixel determines the second boundary segmentation ginseng of the pixel Number, including：To any one pixel, the depth value absolute difference between the pixel and each neighbouring pixel is tired out Plus, then to adding up and being normalized, cumulative and as the pixel the second boundary segmentation after being normalized Parameter；

Wherein, the number range of depth value can be：0~255；

Wherein, the first area determined based on the cromogram partitioning parameters are determined with based on the depth map Second area partitioning parameters merged, including：

Wherein, the first border item partitioning parameters determined based on the cromogram are determined with based on the depth map The second boundary partitioning parameters merged, including：

Wherein, the weight a is determined according to self-evaluation parameter k1 and parameter of consistency k2：By self-evaluation parameter k1 and The product of parameter of consistency k2 is used as weight a；

Wherein, the parameter of consistency k2 is determined in the following ways：

Step S507, builds non-directed graph, by the partitioning parameters mapping after the fusion of each pixel in the mask artwork To in the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask after fine segmentation Figure；

Wherein, non-directed graph sets two hanging point Q as shown in Fig. 5-e outside the plane of the non-directed graph₀And Q₁, it is described outstanding Null point Q₀It is virtual foreground point, the hanging point Q₁It is virtual background point；The mask artwork is set up in the plane of the non-directed graph The mapping point of upper each pixel, in mapping point and the hanging point Q of foreground point₀Between set up line, in the mapping of background dot Point and the hanging point Q₁Between set up line；

The non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtains the mask artwork after fine segmentation, bag Include：

Wherein, step C and D steps include following treatment respectively：

Wherein, the mask artwork after fine segmentation is as shown in Fig. 5-f, comprising sketching the contours track and with target image (stapler) The irregular darker regions of profile are mark zones, and the mark zone is the foreground point block in the mask artwork after fine segmentation；It is former The part that mark zone (foreground point block) is removed on beginning image is the background dot block in mask artwork.Mask artwork after fine segmentation Compared with initial mask artwork, the border between foreground point block and background dot block is more careful.

Step S508, is partitioned into the corresponding figure in foreground point in the mask artwork after the fine segmentation from the cromogram Picture.

Wherein, destination object is partitioned into from the original cromogram according to the mask artwork after fine segmentation, is partitioned into " stapler " image as shown in Fig. 5-g.

The method of present invention application example, when it is closed curve to sketch the contours track, by the way that the region that track is closed will be sketched the contours Used as mark zone, all pixels in mark zone are labeled as foreground point, in the automatic input mask artwork for expanding image segmentation algorithm The prospect of mark is counted out, and it is the iterations for distinguishing foreground point and background dot that can reduce image segmentation algorithm, is significantly subtracted The run time of few image segmentation algorithm.On the other hand, the technical scheme of this application example can be based on depth map and cromogram Calculate respectively each pixel partitioning parameters go forward side by side line parameter fusion, carry out image segmentation using the partitioning parameters after fusion, compare Cromogram is simply used in correlation technique carries out image segmentation, can improve the effect of image segmentation.

Using example 2

Step S601, detects user's selection destination object is marked by the way of sketching the contours；

Step S602, detects user and is sketched the contours on the original image；

Such as, as shown in Fig. 6-a, user is sketched the contours on the original image, and destination object is " stapler ", the hook Strangle track unclosed, described the distance between beginning and end of track of sketching the contours is less than threshold value.

Step S603, mark zone is formed by described sketching the contours after track carries out Seal treatment；

Wherein, if described the distance between beginning and end of track of sketching the contours is less than threshold value, at the starting point and end Line segment connection is done between point, the closure of track is sketched the contours in completion, using closure after sketch the contours the region closed track as mark zone；

Such as, as shown in Fig. 6-b, closing is formed after to connecting line segment between the beginning and end for sketching the contours track bent Line, the region that the curve after the closing is closed is defined as mark zone；

Step S604, the input mask artwork of generation image segmentation algorithm (GrabCut algorithms)：To own in the mark zone Pixel as the foreground point in mask artwork, using the pixel outside mark zone on the original image as the background in mask artwork Point.

Such as, as shown in Fig. 6-c, the irregular darker regions for sketching the contours track superposition line segment rear enclosed are mark zones, described Mark zone is the foreground point block in mask artwork (input mask artwork)；The edge of original image is indicated with dotted line frame, described The part that mark zone (foreground point block) is removed in dotted line frame is the background dot block in mask artwork.

Step S605, obtains cromogram, and the depth map comprising destination object depth information；

Step S606, the first segmentation ginseng of each pixel on the mask artwork is determined according to the cromogram and mask artwork Number, the second partitioning parameters of each pixel on the mask artwork, first segmentation are determined according to the depth map and mask artwork Parameter and the second partitioning parameters are used to represent that pixel judgement is foreground point or the probability and the pixel and adjacent picture of background dot The numerical value difference of element；First partitioning parameters are merged with second partitioning parameters；

Wherein, the first partitioning parameters, second partitioning parameters of each pixel on the mask artwork, and parameter are specifically calculated Correlation technique described in the step of method of fusion is with application example 1 S506 is identical；

Step S607, builds non-directed graph, by the partitioning parameters mapping after the fusion of each pixel in the mask artwork To in the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask after fine segmentation Figure；

Wherein, it is specific to build non-directed graph and partitioning parameters are mapped to the method in the non-directed graph, and according to most It is small to cut the-method that is processed the non-directed graph of maximum-flow algorithm, related side the step of with application example 1 described in S507 Method is identical；

Step S608, is partitioned into covering after the fine segmentation from the original image that the destination object is obtained is shot The corresponding image in foreground point in mould figure；

Wherein, the image of " stapler " is partitioned into from the original cromogram according to the mask artwork after fine segmentation.

The method of present invention application example, when not closed track and smaller breach is sketched the contours, adds by sketch the contours track Line segment closes it, using closure after sketch the contours track closing region as mark zone, all pixels in mark zone are labeled as Foreground point, the prospect marked in the automatic input mask artwork for expanding image segmentation algorithm is counted out, and can reduce image segmentation algorithm To distinguish the iterations of foreground point and background dot, the run time of image segmentation algorithm is significantly decreased.On the other hand, originally Depth map and cromogram can be based on using the technical scheme of example and calculate the partitioning parameters line parameter of going forward side by side of each pixel respectively melting Close, image segmentation is carried out using the partitioning parameters after fusion, image point is carried out compared to cromogram is simply used in correlation technique Cut, the effect of image segmentation can be improved.

Using example 3

Step S701, detects user's selection destination object is marked by the way of sketching the contours；

Step S702, detects user and is sketched the contours on the original image；

Such as, as shown in Fig. 7-a-1, user has carried out one kind and has sketched the contours on the original image, and destination object is " stapler ", Described to sketch the contours that track is unclosed, described the distance between beginning and end of track of sketching the contours is more than or equal to threshold value；

Such as, as shown in Fig. 7-a-2, user has carried out another kind and has sketched the contours on the original image, and destination object is " stapler Machine ", described to sketch the contours that track is unclosed, described the distance between beginning and end of track of sketching the contours is more than or equal to threshold value.

Step S703, mark zone is formed by described sketching the contours after track carries out Seal treatment；

Wherein, if described the distance between beginning and end of track of sketching the contours obtains described more than or equal to threshold value The edge line in the region between the beginning and end of track is sketched the contours, the track of sketching the contours is superimposed with the edge line；If It is described sketch the contours track and be superimposed with the edge line can form closed area, then judge to close successfully, by closing after sketch the contours rail The region that mark is closed is defined as mark zone；If it is described sketch the contours track and be superimposed with the edge line can not form enclosed area Domain, then judge closing failure, and expansion process is carried out to the track of sketching the contours, and the region after the expansion is defined as into mark zone；

Such as, as shown in Fig. 7-b-1, shown in Fig. 7-a-1 sketch the contours track on the basis of, sketch the contours track by described Upper overlay edge line can form closed curve, and the region that the curve after the closing is closed is defined as mark zone；

Such as, as shown in Fig. 7-b-2, shown in Fig. 7-a-2 sketch the contours track on the basis of, sketch the contours track by described Upper overlay edge line can not form closed curve, then expansion process is carried out to the track of sketching the contours, by the area after the expansion Domain is defined as mark zone；

Step S704, the input mask artwork of generation image segmentation algorithm (GrabCut algorithms)：To own in the mark zone Pixel as the foreground point in mask artwork, using the pixel outside mark zone on the original image as the background in mask artwork Point.

Step S705, obtains cromogram, and the depth map comprising destination object depth information；

Step S706, the first segmentation ginseng of each pixel on the mask artwork is determined according to the cromogram and mask artwork Number, the second partitioning parameters of each pixel on the mask artwork, first segmentation are determined according to the depth map and mask artwork Parameter and the second partitioning parameters are used to represent that pixel judgement is foreground point or the probability and the pixel and adjacent picture of background dot The numerical value difference of element；First partitioning parameters are merged with second partitioning parameters；

Step S707, builds non-directed graph, by the partitioning parameters mapping after the fusion of each pixel in the mask artwork To in the non-directed graph, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask after fine segmentation Figure；

Step S708, is partitioned into covering after the fine segmentation from the original image that the destination object is obtained is shot The corresponding image in foreground point in mould figure；

The method of present invention application example, when not closed track and larger breach is sketched the contours, adds by sketch the contours track Edge line closes it, using closure after the region for sketching the contours track closing as mark zone, all pixels mark in mark zone It is foreground point；Or, sketching the contours that track is not closed and breach is larger and when cannot be closed by adding edge line, by that will sketch the contours The region that track is expanded into is labeled as foreground point as mark zone, all pixels in mark zone；The automatic image segmentation that expands is calculated The prospect marked in the input mask artwork of method is counted out, can reduce image segmentation algorithm be distinguish foreground point and background dot repeatedly Generation number, significantly decreases the run time of image segmentation algorithm.On the other hand, the technical scheme of this application example can be based on Depth map and cromogram calculate respectively each pixel partitioning parameters go forward side by side line parameter fusion, carried out using the partitioning parameters after fusion Image segmentation, image segmentation is carried out compared to cromogram is simply used in correlation technique, can improve the effect of image segmentation.

It should be noted that herein, term " including ", "comprising" or its any other variant be intended to non-row His property is included, so that process, method, article or device including a series of key elements not only include those key elements, and And also include other key elements being not expressly set out, or also include for this process, method, article or device institute are intrinsic Key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that including institute Also there is other identical element in process, method, article or the device of stating key element.

The embodiments of the present invention are for illustration only, and the quality of embodiment is not represented.

Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases The former is more preferably implementation method.Based on such understanding, technical scheme is substantially done to prior art in other words The part for going out contribution can be embodied in the form of software product, and the computer software product storage is in a storage medium In (such as ROM/RAM, magnetic disc, CD), including some instructions are used to so that a station terminal equipment (can be mobile phone, computer, clothes Business device, air-conditioner, or network equipment etc.) perform method described in each embodiment of the invention.

The preferred embodiments of the present invention are these are only, the scope of the claims of the invention is not thereby limited, it is every to utilize this hair Equivalent structure or equivalent flow conversion that bright specification and accompanying drawing content are made, or directly or indirectly it is used in other related skills Art field, is included within the scope of the present invention.

Claims

1. a kind of method for realizing interactive image segmentation, including：

Detect on original image sketch the contours track after, described sketching the contours after track carries out Seal treatment is formed into mark zone, generate The input mask artwork of image segmentation algorithm：Using all of pixel in the mark zone as the foreground point in the mask artwork, will Pixel on the original image outside mark zone is used as the background dot in the mask artwork；

The cromogram comprising destination object colouring information and the depth map comprising destination object depth information are obtained, according to the coloured silk Chromatic graph and mask artwork determine the first partitioning parameters of each pixel on the mask artwork, are determined according to the depth map and mask artwork Second partitioning parameters of each pixel on the mask artwork, first partitioning parameters and the second partitioning parameters are used to represent pixel Adjudicate the numerical value difference of the probability and the pixel and adjacent pixel for foreground point or background dot；By first partitioning parameters Merged with second partitioning parameters；

Non-directed graph is built, the partitioning parameters after the fusion of each pixel in the mask artwork are mapped to the non-directed graph In, the non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtain the mask artwork after fine segmentation；

2. method according to claim 1, it is characterised in that：

The first partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork, including：According to institute State cromogram and mask artwork determines the first area partitioning parameters of each pixel on the mask artwork：

Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps；Iteration runs E steps With M steps, the iterative process is stopped after iteration operation reaches the condition of convergence；Last time is performed into M steps to obtain The classification of the pixel be defined as the classification of the pixel, last time is performed the pixel that M steps obtain and belongs to The most probable value P of certain cluster_maxIt is defined as the first area partitioning parameters of the pixel, the first area segmentation ginseng Number is the probability that the pixel is based on that cromogram judgement is foreground point or background dot；

Wherein, the E steps and M steps include following treatment respectively：

E steps：The position relationship between color value and pixel according to each pixel on the mask artwork will be same by clustering The pixel of type is polymerized to one or more clusters, determines the GMM model of each cluster；Wherein, the classification of pixel include foreground point or Background dot；The classification of cluster includes foreground point cluster or background dot cluster；

M steps：GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one pixel, Most probable value P according to the pixel_maxCorresponding cluster determines the classification of the pixel；

The second partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, is also included：According to The depth map and mask artwork determine the second area partitioning parameters of each pixel on the mask artwork：

Gauss hybrid models GMM calculating is carried out by EM methods, the EM methods include E steps and M steps；Iteration runs E steps With M steps, the iterative process is stopped after iteration operation reaches the condition of convergence；Last time is performed into M steps to obtain The classification of the pixel be defined as the classification of the pixel, last time is performed the pixel that M steps obtain and belongs to The most probable value P of certain cluster_maxIt is defined as the second area partitioning parameters of the pixel, the second area segmentation ginseng Number is the probability that the pixel is based on that depth map judgement is foreground point or background dot；

Wherein, the E steps and M steps include following treatment respectively：

E steps：The position relationship between depth value and pixel according to each pixel on the mask artwork will be same by clustering The pixel of type is polymerized to one or more clusters, determines the GMM model of each cluster；Wherein, the classification of pixel include foreground point or Background dot；The classification of cluster includes foreground point cluster or background dot cluster；

M steps：GMM model according to each cluster determines that each pixel belongs to the probability of each cluster, to any one pixel, Most probable value P according to the pixel_maxCorresponding cluster determines the classification of the pixel.

3. method according to claim 2, it is characterised in that：

The first partitioning parameters of each pixel on the mask artwork are determined according to the cromogram and mask artwork, is also included：According to The cromogram and mask artwork determine the first border item partitioning parameters of each pixel on the mask artwork：

Wherein, to any one pixel, by the numerical value on tri- Color Channels of RGB between the pixel and each neighbouring pixel Absolute difference is added up, then to adding up and being normalized, adding up and as the pixel after being normalized The first border item partitioning parameters；

The second partitioning parameters of each pixel on the mask artwork are determined according to the depth map and mask artwork, is also included：According to The depth map and mask artwork determine the second boundary partitioning parameters of each pixel on the mask artwork：

Wherein, to any one pixel, the depth value absolute difference between the pixel and each neighbouring pixel is carried out It is cumulative, then to adding up and being normalized, cumulative and as the pixel the second boundary point after being normalized Cut parameter.

4. method according to claim 3, it is characterised in that：

First partitioning parameters are merged with second partitioning parameters, including：By the first area segmentation ginseng Number is merged with the second area partitioning parameters：

To any one pixel, the first area partitioning parameters are multiplied by the first area after weight (1-a) is adjusted The second area partitioning parameters are multiplied by a second area after weight a is adjusted partitioning parameters by item partitioning parameters；

If the picture that the pixel classifications that the first area partitioning parameters are indicated are indicated with the second area partitioning parameters Element classification is identical, then by the second area partitioning parameters after the first area partitioning parameters after the adjustment and the adjustment Be added and as merge after area item partitioning parameters；

If the picture that the pixel classifications that the first area partitioning parameters are indicated are indicated with the second area partitioning parameters Element classification is differed, then the first area partitioning parameters after the adjustment are split into ginseng with the second area after the adjustment The pixel classifications that numerical value the greater is indicated in number divide the first area after the adjustment as the final classification of the pixel Cut parameter and split ginseng as the area item after merging with the absolute value of the difference of the second area partitioning parameters after the adjustment Number；

First partitioning parameters are merged with second partitioning parameters, is also included：By first border item segmentation Parameter is merged with the second boundary partitioning parameters：

The first border item partitioning parameters are multiplied by the first border item partitioning parameters after weight (1-a) is adjusted, by institute State the second boundary partitioning parameters and be multiplied by a second boundary after weight a is adjusted partitioning parameters, then by after the adjustment The first border item partitioning parameters be added with the second boundary partitioning parameters after the adjustment and obtain the merging of the pixel after Border item partitioning parameters；A is more than or equal to 0 and less than or equal to 1.

5. method according to claim 4, it is characterised in that：

The weight a is determined according to self-evaluation parameter k1 and parameter of consistency k2：Self-evaluation parameter k1 and uniformity are joined The product of number k2 is used as weight a；

Wherein, self-evaluation parameter k1 is determined in the following ways：Depth value according to pixel determines the corresponding shooting of pixel The how far of distance, the how far according to the shooting distance sets self-evaluation parameter k1, and the shooting distance is nearer, from Evaluating k1 sets bigger；K1 is more than or equal to 0 and less than or equal to 1；

Wherein, the parameter of consistency k2 is determined in the following ways：

If the first border item partitioning parameters are equal with the second boundary partitioning parameters, parameter of consistency k2 is set It is first constant；

If the first border item partitioning parameters are unequal with the second boundary partitioning parameters, in first border item When partitioning parameters are simultaneously greater than threshold value or are less than threshold value simultaneously with the second boundary partitioning parameters, parameter of consistency k2 is set It is first constant；Threshold value or not is not simultaneously greater than in the first border item partitioning parameters and the second boundary partitioning parameters When simultaneously less than threshold value, it is second constant to set parameter of consistency k2；The first constant is more than the second constant；Described , more than 0 and less than or equal to 1, the second constant is more than 0 and less than 1 for one constant.

6. method according to claim 1, it is characterised in that：

Build non-directed graph and the partitioning parameters after the fusion of each pixel in the mask artwork are mapped to the non-directed graph In, including：

A non-directed graph is built, two hanging point Q are set outside the plane of the non-directed graph₀And Q₁, the hanging point Q₀For virtual Foreground point, the hanging point Q₁It is virtual background point；Each pixel on the mask artwork is set up in the plane of the non-directed graph Mapping point, in mapping point and the hanging point Q of foreground point₀Between set up line, it is outstanding with described in the mapping point of background dot Null point Q₁Between set up line；

To any one pixel P in mask artwork_i, by the pixel P_iFusion after area item partitioning parameters as the nothing To mapping point P in figure_i' weights, by the pixel P_iFusion after border item partitioning parameters as in the non-directed graph map Point P_i' with hanging point Q₀Or Q₁Between line weights；

The non-directed graph is processed according to minimal cut-maximum-flow algorithm, obtains the mask artwork after fine segmentation, including：

Iteration performs following step C and D steps, stops iterative process after iteration operation reaches the condition of convergence, by the prospect Each pixel in point set Q is used as the foreground point in the mask artwork after fine segmentation；

Wherein, step C and D steps include following treatment respectively：

Step C：One part of pixel in non-directed graph is divided into and vacantly point Q₀Similar foreground point, by being divided into foreground point Pixel constitutes prospect point set Q；

D steps：The weights summation of the prospect point set Q is calculated, before the weights summation is all in the prospect point set Q The weights at sight spot and, along with all foreground points in the prospect point set Q and hanging point Q₀Between line weights and；

7. method according to claim 1, it is characterised in that：

It is described that described sketching the contours after track carries out Seal treatment is formed into mark zone, including：

It is described sketch the contours track it is not closed when, Seal treatment is done to the track of sketching the contours, if closed successfully, after closing Sketch the contours the region closed track and be defined as mark zone, if closing is unsuccessful, expansion process is carried out to the track of sketching the contours, Region after the expansion is defined as mark zone.

8. method according to claim 7, it is characterised in that：

It is described that Seal treatment is done to the track of sketching the contours, including：

If described the distance between beginning and end of track of sketching the contours is more than or equal to threshold value, obtain described in sketch the contours track The edge line in region between beginning and end, the edge line is superimposed with by the track of sketching the contours；If described sketch the contours rail Mark is superimposed with the edge line and can form closed area, then judge close successfully, if described sketch the contours described in track is superimposed with Edge line can not form closed area, then judge closing failure；

If described the distance between beginning and end of track of sketching the contours makees line less than threshold value between the beginning and end The closure of track is sketched the contours in section connection, completion.

9. a kind of device for realizing interactive image segmentation, including：

Pretreatment module, for detect on original image sketch the contours track after, sketched the contours described after track carries out Seal treatment Mark zone is formed, the input mask artwork of image segmentation algorithm is generated：Using all of pixel in the mark zone as the mask Foreground point in figure, using the pixel outside mark zone on the original image as the background dot in the mask artwork；

Partitioning parameters are calculated and Fusion Module, the cromogram of destination object colouring information are included for obtaining and comprising destination object The depth map of depth information, the first segmentation ginseng of each pixel on the mask artwork is determined according to the cromogram and mask artwork Number, the second partitioning parameters of each pixel on the mask artwork, first segmentation are determined according to the depth map and mask artwork Parameter and the second partitioning parameters are used to represent that pixel judgement is foreground point or the probability and the pixel and adjacent picture of background dot The numerical value difference of element；First partitioning parameters are merged with second partitioning parameters；

Mask artwork adjusting module, for building non-directed graph, by the segmentation ginseng after the fusion of each pixel in the mask artwork Number is mapped in the non-directed graph, and the non-directed graph is processed according to minimal cut-maximum-flow algorithm, after obtaining fine segmentation Mask artwork；

Output module, for being partitioned into the corresponding figure in foreground point in the mask artwork after the fine segmentation from the cromogram Picture.

10. the device for realizing interactive image segmentation described in a kind of terminal, including the claims 9.