CN106611422B - Stochastic gradient Bayes's SAR image segmentation method based on sketch structure - Google Patents

Stochastic gradient Bayes's SAR image segmentation method based on sketch structure Download PDF

Info

Publication number
CN106611422B
CN106611422B CN201611262026.9A CN201611262026A CN106611422B CN 106611422 B CN106611422 B CN 106611422B CN 201611262026 A CN201611262026 A CN 201611262026A CN 106611422 B CN106611422 B CN 106611422B
Authority
CN
China
Prior art keywords
sketch
line
sar image
pixel
indicate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611262026.9A
Other languages
Chinese (zh)
Other versions
CN106611422A (en
Inventor
刘芳
李婷婷
袁月
焦李成
郝红侠
尚荣华
马文萍
马晶晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN201611262026.9A priority Critical patent/CN106611422B/en
Publication of CN106611422A publication Critical patent/CN106611422A/en
Application granted granted Critical
Publication of CN106611422B publication Critical patent/CN106611422B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing
    • G06T2207/10044Radar image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning

Landscapes

  • Radar Systems Or Details Thereof (AREA)

Abstract

Stochastic gradient Bayes's SAR image segmentation method based on sketch structure that the invention discloses a kind of mainly solves the problems, such as prior art synthetic aperture radar SAR image segmentation inaccuracy.Implementation step is as follows: (1) SAR image sketch, obtains the sketch map of SAR image;(2) according to the administrative division map of SAR image, the pixel subspace of SAR image is divided;(3) divide the mixing aggregated structure atural object pixel subspace of the method based on stochastic gradient variation Bayesian network model;(4) gather the pinpoint target segmentation of feature based on sketch line;(5) the line target segmentation of view-based access control model semantic rules;(6) the homogenous region pixel subspace segmentation based on multinomial logistic regression prior model;(7) combination and segmentation is as a result, obtain the segmentation result of SAR image.Present invention obtains the good segmentation effects of SAR image, can be used for the semantic segmentation of SAR image.

Description

Stochastic gradient Bayes's SAR image segmentation method based on sketch structure
Technical field
The invention belongs to technical field of image processing, further relate to one of technical field of image segmentation and are based on element Retouch image segmentation side synthetic aperture radar SAR (Synthetic Aperture Radar) of the stochastic gradient Bayes of structure Method.Present invention can apply to the different zones to synthetic aperture radar SAR to be accurately split, and can be further used for SAR figure Object detection and recognition as in.
Background technique
Synthetic aperture radar SAR is the impressive progress in remote sensing technology field, for obtaining the full resolution pricture of earth surface. Compared with other kinds of imaging technique, SAR has very important advantage, it is not by atmosphere such as cloud layer, rainfall or dense fogs The influence of condition and intensity of illumination, can round-the-clock, round-the-clock obtain high resolution remote sensing data.SAR technology for it is military, Many fields such as agricultural, geography have great importance.
Image segmentation, which refers to, divides an image into several mutually disjoint regions according to color, gray scale and Texture eigenvalue Process.The common method of image segmentation has at present: the method based on edge detection, the method based on threshold value are based on region life Long and watershed method and the method based on cluster etc..Due to the unique imaging mechanism of SAR, containing there are many phases in SAR image Dry spot noise causes the conventional method of many optical imagerys all to cannot be directly used to the segmentation of SAR image.The tradition of SAR image Dividing method includes some methods based on cluster such as K-means, FCM and some other has supervision and semi-supervised side Method.They generally require manually experience and carry out feature extraction, however the quality for the feature extracted is for the segmentation knot of SAR image Fruit has an important influence.For having supervision and semi-supervised method, label data is needed, the label data of SAR image is seldom, The cost for obtaining label data is very high.Bayesian network has unique advantage in terms of the expression of uncertainty knowledge and reasoning, Variation Bayesian inference network both may not need label data carry out it is unsupervisedly trained, can also effectively learn each pixel The implicit structure feature in space has great significance for effective segmentation of SAR image.
Paper " a kind of effective MSTAR SAR image segmentation method " that Wuhan University delivers at it (Wuhan University Journal: Information science version page 1377-page 1380 of in October, 2015) in propose a kind of MSTAR SAR image segmentation method.This method Over-segmentation operation is carried out to image to be processed first, segmented image region is obtained.Secondly figure is carried out to the image after over-segmentation As the feature extraction of region class and Pixel-level, the feature vector for indicating image is obtained, to MSTARSAR image use space Latent dirichlet allocation model (sLDA) and markov random file (MRF) establish the model that this method is proposed, obtain energy Measure functional.Finally energy functional is optimized with Graph-Cut algorithm and Branch-and-Bound algorithm, is obtained final Segmentation result.Shortcoming existing for this method is, in the feature vector for acquiring SAR image, uses the pixel of SAR image Grade feature, without automatically go study SAR image in due to the correlation between pixel distinctive structure feature so that very It is insufficient just to indicate that the structure feature of SAR image atural object feature utilizes, is causing segmentation result not accurate enough.
Patent " SAR image based on deconvolution network Yu mapping inference network of the Xian Electronics Science and Technology University in its application A kind of deconvolution net is disclosed in dividing method " (number of patent application CN201510679181.X, publication number CN105389798A) The SAR image segmentation method of network and mapping inference network.This method is drawn according to the sketch map of synthetic aperture radar SAR image Point administrative division map, by administrative division map be mapped to original image assembled, homogeneous and structural region.Respectively to each in aggregation and homogenous region A mutually disconnected region carries out unsupervised training, obtains the filter set for characterizing each mutual not connected region structure feature. Reasoning is compared between the structure feature not connected region mutual in two class regions respectively, obtains point of aggregation and homogenous region Cut result.Structural region is split under the guidance of sketch line segment using super-pixel merging.It is complete to merge each region segmentation result Divide at SAR image.Shortcoming existing for this method is, between in aggregation zone mutually not the structure feature connected region into When row compares reasoning, the inference network that this method uses is self-organizing feature map SOM network, due to Self-organizing Maps SOM itself has artificial determination cluster numbers, clusters disadvantage of long duration, and SOM is incited somebody to action when to the processing of SAR filter characteristic Filter characteristic cluster with obvious direction difference is one kind, causes cluster accuracy greatly to reduce, greatly affected SAR The accuracy of image segmentation.
Patent " SAR image based on deconvolution network Yu mapping inference network of the Xian Electronics Science and Technology University in its application A kind of deconvolution net is disclosed in dividing method " (number of patent application CN201510679181.X, publication number CN105389798A) The SAR image segmentation method of network and mapping inference network.This method is drawn according to the sketch map of synthetic aperture radar SAR image Point administrative division map, by administrative division map be mapped to original image assembled, homogeneous and structural region.Respectively to each in aggregation and homogenous region A mutually disconnected region carries out unsupervised training, obtains the filter set for characterizing each mutual not connected region structure feature. Reasoning is compared between the structure feature not connected region mutual in two class regions respectively, obtains point of aggregation and homogenous region Cut result.Structural region is split under the guidance of sketch line segment using super-pixel merging.It is complete to merge each region segmentation result Divide at SAR image.Shortcoming existing for this method is, between in aggregation zone mutually not the structure feature connected region into When row compares reasoning, for this method using the inference method of self-organizing feature map SOM network, this inference method need to Very important person is to determine cluster numbers, and the cluster time is more long, and cluster accuracy is caused to reduce, and affects the accuracy of SAR image segmentation.
Liu Fang, Duan Yiping, Li Lingling, burnt Lee " are based on the semantic and adaptive neighbour of level vision in its paper delivered at equal The SAR image of the hidden model of domain multinomial is divided " (IEEE Trancactions on Geoscience and Remote Sensing, 2016,54 (7): 4287-4301.) in propose it is a kind of based on level vision semanteme and adaptive neighborhood multinomial The SAR image segmentation method of hidden model, this method propose the level vision of SAR image on the basis of SAR image sketch map It is semantic.SAR image is divided into aggregation zone, structural region and homogenous region by the level vision semanteme.Based on the division, to not Region with characteristic uses different dividing methods.For aggregation zone, gray level co-occurrence matrixes feature is extracted, and using part The method of linear restriction coding obtains the expression of each aggregation zone, and then is split using the method for hierarchical clustering.To knot Structure region devises vision semantic rules positioning boundary and line target by analysis side mode type and line model.In addition, boundary and Line target contains strong directional information, therefore devises the hidden model of multinomial based on geometry window and be split.It is right Homogenous region is gone to indicate center pixel, devises the hidden mould of multinomial based on self-adapting window in order to find appropriate neighborhood Type is split.The segmentation result in these three regions be integrated into together segmentation result to the end.The deficiency of this method Place is, inaccurate to the boundary alignment of aggregation zone, not reasonable to the determination of homogenous region classification number, the area of segmentation result Domain consistency is poor, and does not handle pinpoint target in the segmentation of structural region.
Summary of the invention
The present invention proposes one kind in conjunction with the advantage of variation Bayesian inference network for the deficiency of above-mentioned prior art Stochastic gradient Bayes's SAR image segmentation method based on sketch structure, to improve the accuracy of SAR image segmentation.
To achieve the above object, technical scheme is as follows:
(1) SAR image sketch:
(1a) inputs synthetic aperture radar SAR image;
(1b) establishes the sketch model of synthetic aperture radar SAR image;
The sketch map of (1c) from sketch model extraction synthetic aperture radar SAR image;
(2) pixel subspace is divided:
(2a) uses sketch line fields method, carries out compartmentalization processing to the sketch map of synthetic aperture radar SAR image, Obtain include aggregation zone, synthetic aperture radar SAR image without sketch line region and structural region administrative division map;
(2b) will include aggregation zone, the administrative division map without sketch line region and structural region, be mapped to synthetic aperture radar In SAR image, mixing aggregated structure atural object pixel subspace, homogenous region pixel of synthetic aperture radar SAR image are obtained Space and structure-pixel subspace;
(3) stochastic gradient variation Bayesian network model is constructed:
(3a) according to the following formula, calculates the input layer of stochastic gradient variation Bayesian network model to the intermediate variable of hidden layer:
Wherein, hφIndicate stochastic gradient variation Bayesian network model input layer to hidden layer intermediate variable,Table Show the input layer of stochastic gradient variation Bayesian network model to intermediate variable hφConnection weight, m indicate hidden layer neuron Number, the neuron number of m=500, n expression input layer, n=441,It indicatesCorresponding bias vector;
(3b) according to the following formula, calculates the approximate posterior probability of stochastic gradient variation Bayesian network model:
Wherein, qφ(z | x) indicates the approximate posterior probability of stochastic gradient variation Bayesian network model, Expression mean vector is μφ, covariance matrix isNormal distribution,Indicate stochastic gradient variation Bayesian network mould Intermediate variable h of the input layer of type to hidden layerφWith μφConnection weight,It indicatesCorresponding bias vector,Table Show input layer to hidden layer intermediate variable hφWith σφConnection weight,It indicatesCorresponding bias vector;
(3c) according to the following formula, calculates the hidden layer of stochastic gradient variation Bayesian network model to the intermediate variable of reconstruction of layer:
Wherein, hθIndicate stochastic gradient variation Bayesian network model hidden layer to reconstruction of layer intermediate variable,It is hidden Layer arrives intermediate variable hθConnection weight,It indicatesCorresponding bias vector;
(3d) according to the following formula, calculates the conditional probability of stochastic gradient variation Bayesian network model:
Wherein, pθ(y | z) indicates the conditional probability of stochastic gradient variation Bayesian network model,It indicates Mean vector is μθ, covariance matrix isNormal distribution,Intermediate variable h of the expression hidden layer to reconstruction of layerθWith μθ Connection weight,It indicatesCorresponding bias vector,Intermediate variable h of the expression hidden layer to reconstruction of layerθWith σθCompany Weight is connect,It indicatesCorresponding bias vector;
(3e) according to the following formula, calculates the variation lower bound of stochastic gradient variation Bayesian network model:
Wherein, J (θ, φ) indicates the variation lower bound of stochastic gradient variation Bayesian network model,Indicate that stochastic gradient becomes Divide the variational parameter of Bayesian network model, Indicate stochastic gradient variation The generation parameter of Bayesian network model,DKL(qφ(z|x)||pθ(z)) it indicates qφ(z | x) and pθ(z) relative entropy between, z indicate the hidden layer variable of stochastic gradient variation Bayesian network model, pθ(z) it indicates The prior probability of hidden layer variable z, ∑ () indicate sum operation, and L indicates that hidden layer variable z carries out the number of Gauss sampling, log () indicates log operations, zlIndicate that value is by formula to the l times Gauss sampled result of zIt obtains, Wherein, ⊙ indicates point multiplication operation, εlIndicate that Gauss samples auxiliary variable, εl~N (0, I) indicates that Gauss samples auxiliary variable and meets Standardized normal distribution;
(3f) according to the following formula, calculates the structural remodeling error of stochastic gradient variation Bayesian network model:
Wherein, G (θ, φ) indicates the structural remodeling error of stochastic gradient variation Bayesian network model, and K indicates input figure As the sum of block, xiIndicate i-th of input picture block, yiIndicate xiReconstructed image block, SM () expression ask sketch block operate, C () indicates to ask sketch line length in sketch block to operate;
(4) feature learning is carried out to mixing aggregated structure atural object pixel subspace:
(4a) to the mixing aggregated structure atural object pixel subspace of synthetic aperture radar SAR image, spatially on connection Property carry out region division, if only existing a mutually not connected region, execute (6);
(4b) to each mutual not connected region is carried out that it is corresponding to obtain each region every a sampling by 21 × 21 window Multiple images block sample;
(4c) takes and the one-to-one sketch block sample of image block sample each image block sample in sketch map;
(4d) produces corresponding one group of each region and meets uneven G0 points of atural object distribution to each mutual not connected region The random number of cloth;
(4e) to each mutually not connected region, with the corresponding one group of random number in each region to stochastic gradient variation Bayes The connection weight of network is initialized, the stochastic gradient variation Bayesian network after being initialized;
Image block sample is made the variation Bayesian network of gradient immediately after each mutual not connected region initialization by (4f) For the input layer of stochastic gradient variation Bayesian network, using the side of the stochastic gradient variation Bayesian inference of sketch structural constraint Method carries out structural constraint training to the stochastic gradient variation Bayesian network after initialization, and the stochastic gradient after being trained becomes Divide Bayesian network;
(4g) to each mutually not connected region, the weight of the stochastic gradient variation Bayesian network after taking it to train, as The characteristic set in the region;
(5) segmentation SAR image mixes aggregated structure atural object pixel subspace:
(5a) splices the characteristic set of all mutual connected regions, using spliced characteristic set as code book;
(5b) calculates separately the inner product with each feature in code book, obtains to all features of each mutual not connected region To the projection vector of all features on code book in each region;
(5c) it is corresponding to obtain each region to each mutually all projection vectors progress maximum value convergence of connected region One structural eigenvector;
(5d) propagates AP clustering algorithm using neighbour, does not cluster to the structural eigenvector of all mutual connected regions, Obtain the segmentation result of mixing aggregated structure atural object pixel subspace;
(6) segmenting structure pixel subspace:
(6a) uses vision semantic rules, divides line target;
The feature of gathering of (6b) based on sketch line divides pinpoint target;
The result that (6c) divides line target and pinpoint target merges, and obtains the segmentation knot of structure-pixel subspace Fruit;
(7) divide homogenous region pixel subspace:
Using the homogenous region dividing method based on multinomial logistic regression prior model, to homogenous region pixel subspace It is split, obtains the segmentation result of homogenous region pixel subspace;
(8) combination and segmentation result:
The segmentation of aggregated structure atural object pixel subspace, homogenous region pixel subspace and structure-pixel subspace will be mixed As a result merge, obtain the final segmentation result of synthetic aperture radar SAR image.
The invention has the following advantages over the prior art:
First, since the present invention is provided with a kind of stochastic gradient variation Bayesian network, by mixing aggregated structure atural object The each mutually disconnected region in pixel subspace carries out unsupervised training, and the weight that training obtains network is not connected mutually as each The structure feature in logical region, the Pixel-level feature for overcoming prior art SAR image seek the feature vector of SAR image, and Do not learn in SAR image due to the correlation between pixel and the shortcomings that distinctive structure feature so that can using the present invention To automatically extract the structure feature of SAR image, better region consistency is obtained.
Second, since the present invention carries out feature learning in each mutually disconnected region of to mixing aggregated structure object space When, G is estimated with the pixel intensity value of each region respectively0The parameter of distribution, then uses G0The probability density function of distribution generate with Machine data initialize the weight of network, overcome the prior art and automatically extract in the depth autoencoder network of characteristics of image With random distribution to netinit without catching SAR image substantive characteristics the shortcomings that, allow effective using the present invention The substantive characteristics for acquiring characterization SAR image atural object improves the accuracy of SAR image segmentation.
Third, since the present invention schemes SAR using the method for the stochastic gradient variation Bayesian inference of sketch structural constraint It is overcome as carrying out structure feature study using the structural information in SAR image sketch map as the structural constraint of feature learning The prior art, which automatically extracts, does not have the shortcomings that structure feature using SAR image in the depth autoencoder network of characteristics of image, make The most important structure feature that can acquire characterization SAR image atural object using the present invention is obtained, SAR image segmentation is further improved Accuracy.
4th, due to the present invention by it is all mutually it is disconnected mixing aggregated structure atural object pixel subspace regions feature sets It is merged and connects, using spliced characteristic set as code book, and by the way of inner product projection, by the feature in each region to code The structural eigenvector of this projection, obtained each region is sparse and ga s safety degree is big, and cluster efficiency is obviously improved, and overcomes existing Have the shortcomings that technology based on deconvolution network and the artificial determining clusters number in mapping inference network and cluster it is of long duration so that Using the present invention can more accurately obtain SAR image segmentation result and in time improve SAR image segmentation efficiency.
Detailed description of the invention
Fig. 1 is flow chart of the invention;
Fig. 2 is analogous diagram of the invention;
Fig. 3 is emulation experiment intermediate result of the present invention.
Fig. 4 is simulation result schematic diagram of the present invention.
Specific embodiment
The present invention will be further described with reference to the accompanying drawing.
Referring to attached drawing 1, specific implementation step of the invention is as follows:
Step 1, SAR image sketch.
Input synthetic aperture radar SAR image.
According to the following steps, the sketch model of synthetic aperture radar SAR image is established:
Step 1 arbitrarily chooses a number, the sum as template in [100,150] range;
Step 2 constructs a template on the side being made of pixel with different directions and scale, line, utilizes template Direction and dimensional information structural anisotropy's Gaussian function, by the Gaussian function, in calculation template each pixel plus Weight coefficient, the weighting coefficient of all pixels point in statistical mask, wherein scale number value is 3~5, and direction number value is 18;
Step 3 calculates pixel in synthetic aperture radar SAR image corresponding with template area coordinate according to the following formula Mean value:
Wherein, μ indicates the equal of all pixels point in corresponding with template area coordinate synthetic aperture radar SAR image Value, ∑ indicate sum operation, and g indicates the corresponding coordinate of any one pixel in the Ω region of template, and ∈ expression belongs to symbol Number, wgIndicate weight coefficient of the pixel at coordinate g in the Ω region of template, wgValue range be wg∈ [0,1], Ag Indicate the value of the pixel with pixel in the Ω region of template at the coordinate g in corresponding synthetic aperture radar SAR image;
Step 4 calculates pixel in synthetic aperture radar SAR image corresponding with template area coordinate according to the following formula Variance yields:
Wherein, ν indicates the variance of all pixels point in synthetic aperture radar SAR image corresponding with template area coordinate Value;
Step 5 calculates the response that each pixel in synthetic aperture radar SAR image is directed to ratio operator according to the following formula Value:
Wherein, R indicates response of each pixel for ratio operator, min { } in synthetic aperture radar SAR image Indicate minimum Value Operations, a and b respectively indicate two different regions in template, μaIndicate all pixels point in a of template area Mean value, μbIndicate the mean value of all pixels point in the b of template area;
Step 6 calculates the response that each pixel in synthetic aperture radar SAR image is directed to correlation operator according to the following formula Value:
Wherein, C indicate synthetic aperture radar SAR image in each pixel be directed to correlation operator response,It indicates Square root functions, a and b respectively indicate two different zones, ν in templateaIndicate the variance of all pixels point in a of template area Value, νbIndicate the variance yields of all pixels point in the b of template area, μaIndicate the mean value of all pixels point in a of template area, μbTable Show the mean value of all pixels point in the b of template area;
Step 7 calculates the response that each pixel in synthetic aperture radar SAR image is directed to each template according to the following formula Value:
Wherein, F indicate synthetic aperture radar SAR image in each pixel be directed to each template response,It indicates Square root functions, R and C respectively indicate pixel in synthetic aperture radar SAR image and are directed to ratio operator and synthetic aperture radar Pixel is directed to the response of correlation operator in SAR image;
Step 8, judges whether constructed template is equal to the sum of selected template, if so, step 2 is executed, otherwise, Execute step 9;
Step 9, selection has the template of maximum response from each template, as synthetic aperture radar SAR image Template, and using the maximum response of the template as the intensity of pixel in synthetic aperture radar SAR image, by the side of the template The direction of pixel in as synthetic aperture radar SAR image, obtain synthetic aperture radar SAR image sideline response diagram and Gradient map;
Step 10 calculates the intensity value of synthetic aperture radar SAR image intensity map, obtains intensity map according to the following formula:
Wherein, I indicates that the intensity value of synthetic aperture radar SAR image intensity map, r indicate synthetic aperture radar SAR image Value in the response diagram of sideline, t indicate the value in synthetic aperture radar SAR image gradient map;
Step 11 detects intensity map using non-maxima suppression method, obtains suggestion sketch;
Step 12, choose suggest sketch in maximum intensity pixel, will suggest sketch in the maximum intensity The pixel of pixel connection connects to form suggestion line segment, obtains suggestion sketch map;
Step 13 calculates the code length gain for suggesting sketch line in sketch map according to the following formula:
Wherein, CLG indicates to suggest the code length gain of sketch line in sketch map, and ∑ indicates sum operation, and J indicates current The number of pixel, A in sketch line neighborhoodjIndicate the observation of j-th of pixel in current sketch line neighborhood, Aj,0It indicates In the case that current sketch line cannot indicate structural information, the estimated value of j-th of pixel, ln () table in the sketch line neighborhood Show the log operations using e the bottom of as, Aj,1Indicate the sketch line neighborhood in the case where current sketch line can indicate structural information In j-th of pixel estimated value;
Step 14 arbitrarily chooses a number, as threshold value T in [5,50] range;
Step 15 selects the suggestion sketch line of CLG > T in all suggestion sketch lines, is combined into synthetic aperture radar The sketch map of SAR image.
From the sketch map of sketch model extraction synthetic aperture radar SAR image.
The synthetic aperture radar SAR image sketch model that the present invention uses is that Jie-Wu et al. in 2014 was published in IEEE Article " Local maximal on Transactions on Geoscience and Remote Sensing magazine homogenous region search for SAR speckle reduction with sketch-based Geometrical kernel function " proposed in model.
Step 2, pixel subspace is divided.
Using sketch line fields method, compartmentalization processing is carried out to the sketch map of synthetic aperture radar SAR image, is obtained The administrative division map of synthetic aperture radar SAR image including aggregation zone, without sketch line region and structural region.
According to the concentration class of sketch line segment in the sketch map of synthetic aperture radar SAR image, sketch line is divided into expression Assemble the aggregation sketch line of atural object and indicates boundary, line target, the boundary sketch line of isolated target, line target sketch line, isolates Target sketch line.
According to the statistics with histogram of sketch line segment concentration class, the sketch line segment conduct that concentration class is equal to optimal concentration class is chosen Seed line-segment sets { Ek, k=1,2 ..., m }, wherein EkIndicate that any bar sketch line segment in seed line-segment sets, k indicate seed The label of any bar sketch line segment in line-segment sets, m indicate the total number of seed line segment, and { } indicates set operation.
As basic point with the unselected line segment for being added to seed line-segment sets sum, with this basic point recursive resolve line segment aggregate.
The round primitive that a radius is the optimal concentration class section upper bound is constructed, with the circle primitive in line segment aggregate Line segment is expanded, and is corroded to the line segment aggregate ecto-entad after expansion, is obtained as unit of sketch point in sketch map Aggregation zone.
To the sketch line for indicating boundary, line target and isolated target, centered on each sketch point of each sketch line The geometry window that size is 5 × 5 is constructed, structural region is obtained.
The part other than aggregation zone and structural region will be removed in sketch map as can not sketch region.
By in sketch map aggregation zone, can not sketch region and structural region merge, to obtain include aggregation zone, without element Retouch the administrative division map of the synthetic aperture radar SAR image of line region and structural region.
It will include aggregation zone, the administrative division map without sketch line region and structural region, be mapped to synthetic aperture radar SAR figure As in, obtain the mixing aggregated structure atural object pixel subspace of synthetic aperture radar SAR image, homogenous region pixel subspace and Structure-pixel subspace.
Step 3, stochastic gradient variation Bayesian network model is constructed.
According to the following formula, the input layer of stochastic gradient variation Bayesian network model is calculated to the intermediate variable of hidden layer:
Wherein, hφIndicate stochastic gradient variation Bayesian network model input layer to hidden layer intermediate variable,Table Show the input layer of stochastic gradient variation Bayesian network model to intermediate variable hφConnection weight, m indicate hidden layer neuron Number, the neuron number of m=500, n expression input layer, n=441,It indicatesCorresponding bias vector.
According to the following formula, the approximate posterior probability of stochastic gradient variation Bayesian network model is calculated:
Wherein, qφ(z | x) indicates the approximate posterior probability of stochastic gradient variation Bayesian network model, Expression mean vector is μφ, covariance matrix isNormal distribution,Indicate stochastic gradient variation Bayesian network mould Intermediate variable h of the input layer of type to hidden layerφWith μφConnection weight,It indicatesCorresponding bias vector,Table Show input layer to hidden layer intermediate variable hφWith σφConnection weight,It indicatesCorresponding bias vector.
According to the following formula, the hidden layer of stochastic gradient variation Bayesian network model is calculated to the intermediate variable of reconstruction of layer:
Wherein, hθIndicate stochastic gradient variation Bayesian network model hidden layer to reconstruction of layer intermediate variable,It is hidden Layer arrives intermediate variable hθConnection weight,It indicatesCorresponding bias vector.
According to the following formula, the conditional probability of stochastic gradient variation Bayesian network model is calculated:
Wherein, pθ(y | z) indicates the conditional probability of stochastic gradient variation Bayesian network model,It indicates Mean vector is μθ, covariance matrix isNormal distribution,Intermediate variable h of the expression hidden layer to reconstruction of layerθWith μθ Connection weight,It indicatesCorresponding bias vector,Intermediate variable h of the expression hidden layer to reconstruction of layerθWith σθCompany Weight is connect,It indicatesCorresponding bias vector.
According to the following formula, the variation lower bound of stochastic gradient variation Bayesian network model is calculated:
Wherein, J (θ, φ) indicates the variation lower bound of stochastic gradient variation Bayesian network model,Indicate that stochastic gradient becomes Divide the variational parameter of Bayesian network model, Indicate stochastic gradient variation The generation parameter of Bayesian network model,DKL(qφ(z|x)||pθ(z)) it indicates qφ(z | x) and pθ(z) relative entropy between, z indicate the hidden layer variable of stochastic gradient variation Bayesian network model, pθ(z) it indicates The prior probability of hidden layer variable z, ∑ () indicate sum operation, and L indicates that hidden layer variable z carries out the number of Gauss sampling, log () indicates log operations, zlIndicate that value is by formula to the l times Gauss sampled result of zIt obtains, Wherein, ⊙ indicates point multiplication operation, εlIndicate that Gauss samples auxiliary variable, εl~N (0, I) indicates that Gauss samples auxiliary variable and meets Standardized normal distribution.
According to the following formula, the structural remodeling error of stochastic gradient variation Bayesian network model is calculated:
Wherein, G (θ, φ) indicates the structural remodeling error of stochastic gradient variation Bayesian network model, and K indicates input figure As the sum of block, xiIndicate i-th of input picture block, yiIndicate xiReconstructed image block, SM () expression ask sketch block operate, C () indicates to ask sketch line length in sketch block to operate.
Step 4, the feature learning of aggregated structure atural object pixel subspace is mixed.
To the mixing aggregated structure atural object pixel subspace of synthetic aperture radar SAR image, spatially on connectivity into Row region division, if only one mutual not connected region, executes step 6.
To each mutual not connected region, carry out that it is corresponding multiple to obtain each region every a sampling by 21 × 21 window Image block sample.
To each image block sample, taken in sketch map and the one-to-one sketch block sample of image block sample.
To each mutual not connected region, produces corresponding one group of each region and meet uneven atural object distribution G0Distribution Random number.
According to the following formula, it calculates the uneven atural object of synthetic aperture radar SAR image and is distributed G0The probability density of distribution:
Wherein, the probability density of the uneven atural object distribution of P (I (x, y)) expression synthetic aperture radar SAR image, I (x, Y) indicates coordinate is the intensity value of the pixel of (x, y), and n indicates the equivalent number of synthetic aperture radar SAR image, and α indicates to close At the form parameter of aperture radar SAR image, γ indicates that the scale parameter of synthetic aperture radar SAR image, Γ () indicate gal Horse function, value are obtained by following formula:
Wherein, u indicates that independent variable, ∫ indicate integration operation, and t indicates integration variable.
To mixing aggregated structure atural object pixel subspace region Ri, using the intensity value of all pixels point in the region, adopt With the method for parameter estimation of Mellin transform, the uneven atural object distribution G of synthetic aperture radar SAR image is obtained0Needed for distribution Three parameter alphas, the estimated value of γ, n.
With randomly selecting mixing aggregated structure object space Ri500 image block samples, composition 441 × 500 matrix A.
Using matrix A, G is distributed by the uneven atural object of synthetic aperture radar SAR image0The probability density function of distribution One 441 × 500 matrix B is generated, the data in matrix B meet the uneven atural object distribution of synthetic aperture radar SAR image G0Distribution.
To each mutual not connected region, with the corresponding one group of random number in each region to stochastic gradient variation Bayesian network Connection weight initialized, the stochastic gradient variation Bayesian network after being initialized.
Using matrix B as the input layer x of stochastic gradient variation Bayesian network model to intermediate variable hφConnection weight
500 column, the Matrix C of composition 500 × 500, using Matrix C as stochastic gradient variation shellfish are randomly selected from matrix B The intermediate variable h of this network model of leafφTo μφConnection weightUsing Matrix C as stochastic gradient variation Bayesian network The intermediate variable h of modelφTo σφConnection weightUsing Matrix C as the hidden layer z of stochastic gradient variation Bayesian network model To intermediate variable hθConnection weight
Using the transposition of matrix B as the intermediate variable h of stochastic gradient variation Bayesian network modelθTo μθConnection weightUsing the transposition of matrix B as the intermediate variable h of stochastic gradient variation Bayesian network modelθTo μθConnection weight
To it is each mutually not connected region initialization after the variation Bayesian network of gradient immediately, using image block sample as with The input layer of machine gradient variation Bayesian network, according to the following steps, using the stochastic gradient variation pattra leaves of sketch structural constraint The method of this reasoning carries out structural constraint training to the stochastic gradient variation Bayesian network after initialization, after being trained Stochastic gradient variation Bayesian network:
The prior probability of stochastic gradient variation Bayesian network model hidden layer is initialized as standardized normal distribution by step 1 The approximate posterior probability of stochastic gradient variation Bayesian network model is initialized as normal distribution probability by probability, is obtained random The analytic expression of the variation lower bound of gradient variation Bayesian network model is as follows:
Step 2 updates the generation parameter of stochastic gradient variation Bayesian network model according to the following formula:
Wherein, θt+1Indicate the generation parameter of stochastic gradient variation Bayesian network model after the t+1 times iteration, θtIt indicates The generation parameter of stochastic gradient variation Bayesian network model after the t times iteration,Indicate the parameter to J (θ, φ) θ asks the operation of local derviation;
Step 3 updates the variational parameter of stochastic gradient variation Bayesian network model according to the following formula:
Wherein, φt+1Indicate the variational parameter of stochastic gradient variation Bayesian network model after the t+1 times iteration, φtTable Show the variational parameter of stochastic gradient variation Bayesian network model after the t times iteration,It indicates to become the ginseng of J (θ, φ) Amount φ asks the operation of local derviation;
Step 4 calculates structural remodeling and misses using the structural remodeling error formula of stochastic gradient variation Bayesian network model Difference;
Step 5, judges whether structural remodeling error is less than threshold value 0.2, if so, executing step 5;Otherwise, step 1 is executed;
Step 6 completes the structural constraint training of stochastic gradient variation Bayesian network.
Step 5, segmentation SAR image mixes aggregated structure atural object pixel subspace.
By all mutually not characteristic set splicings of connected region, using spliced characteristic set as code book.
To all features of each mutual not connected region, the inner product with each feature in code book is calculated separately, is obtained every Projection vector of all features in a region on code book.
To each mutually all projection vectors progress maximum value convergence of connected region, it is one corresponding to obtain each region Structural eigenvector.
AP clustering algorithm is propagated using neighbour, the structural eigenvector of all mutual connected regions is not clustered, is obtained Mix the segmentation result of aggregated structure atural object pixel subspace.
Step 6, segmenting structure pixel subspace.
With vision semantic rules, divide line target.
If i-th sketch line liWith j-th strip sketch line ljThe distance between be Dij, liDirection be Oi, ljDirection be Oj, I, j ∈ [1,2 ..., S], S are the total number of sketch line.
Width is greater than the line target of 3 pixels with two sketch line liAnd ljIt indicates, liAnd ljThe distance between DijIt is less than T1And direction difference (Oi- Oj) less than 10 degree, wherein T1=5.
If the s articles sketch line lsGeometry window wsThe average gray of interior each column is AiIf the gray scale difference of adjacent column is ADi=| Ai-Ai+1|, if zs=[zs1,zs2,...,zs9] be adjacent column gray scale difference ADi label vector.
By width less than the line target of 3 pixels with single sketch line lsIt indicates, lsGeometry window wsIt is interior, calculate phase The gray scale difference AD of adjacent columniIf ADi>T2, then zsi=1;Otherwise zsi=0, zsIn there are two element value be 1, remaining is 0, Middle T2=34.
If L1,L2It is the set for indicating the sketch line of line target, if Dij<T1And | Oi- Oj| < 10, then li, lj∈L1;If sum(zs)=2, then ls∈L2, wherein sum () indicate to vector important summation operation.
In structure-pixel subspace, according to the set L1 of the sketch line of line target, using the region between li and lj as line Target.
In structure-pixel subspace, according to the set L2 of the sketch line of line target, using the region of ls as line target.
According to the following steps, based on the feature of gathering of sketch line, divide pinpoint target:
Step 1 would not indicate all sketch wire tags of line target in the structural region of administrative division map as candidate sketch line Sketch line in set;
Step 2 randomly selects a sketch line from candidate sketch line set, with an endpoint of selected sketch line Centered on, construct the geometry window that size is 5 × 5;
Step 3 judges the endpoint that whether there is other sketch lines in geometry window, and if it exists, execute step 4;Otherwise, Execute step 6;
Step 4 judges whether to only exist an endpoint, if so, sketch line where the endpoint and current sketch line are carried out Connection;Otherwise, step 5 is executed;
Step 5 connects the sketch line where selected sketch line and each endpoint, wherein angle is chosen from all connecting lines The sketch line that maximum two sketch lines are completed as connection;
Step 6 judges the endpoint that whether there is other sketch lines in the geometry window of another endpoint of sketch line, if In the presence of execution step 4;Otherwise, step 7 is executed;
Step 7 chooses the sketch line comprising two and two or more sketch line segments to the sketch line for completing attended operation, The item number n comprising sketch line segment in selected sketch line is counted, wherein n >=2;
Step 8, judges whether the item number n of sketch line is equal to 2, if so, executing step 9;Otherwise, step 10 is executed;
Sketch line of the angle value on sketch line vertex in the range of [10 °, 140 °] is used as to have and gathers spy by step 9 The sketch line of sign;
Step 10 selects sketch line of the angle value on the corresponding n-1 vertex of sketch line all in [10 °, 140 °] range;
Step 11 is defined as follows two kinds of situations in selected sketch line:
Whether the first situation judges adjacent (i-1)-th, the two sketch line segments of i-th sketch line segment, i+1 item i-th The same side of straight line, 2≤i≤n-1, if all sketch line segments and adjacent segments on sketch line are all same where sketch line segment Side, then marking the sketch line is with the sketch line for gathering feature;
Whether second situation judges adjacent (i-1)-th, the two sketch line segments of i-th sketch line segment, i+1 item i-th The same side of straight line, 2≤i≤n-1, if having n-1 sketch line segment and adjacent segments same on sketch line where sketch line segment Side, and have a sketch line segment line segment adjacent thereto in non-the same side, also marking the sketch line is with the sketch for gathering feature Line;
Step 12, an optional sketch line in there is the sketch line for gathering feature, by two ends of selected sketch line Point coordinate, determine the distance between two endpoints, if the end-point distances in [0,20] range, then using selected sketch line as table Show the sketch line of pinpoint target;
Step 13, judge it is untreated have gather the sketch line of feature and whether all selected, if so, executing step 12; Otherwise, step 14 is executed;
Step 14, with the method for super-pixel segmentation, to the sketch line for indicating pinpoint target in synthetic aperture radar SAR image The pixel of surrounding carries out super-pixel segmentation, by super-pixel of the gray value of super-pixel after segmentation in [0,45] or [180,255] As pinpoint target super-pixel;
Step 15 merges pinpoint target super-pixel, using the boundary of the pinpoint target super-pixel after merging as pinpoint target Boundary, obtain the segmentation result of pinpoint target.
The result divided to line target and pinpoint target merges, and obtains the segmentation result of structure-pixel subspace.
Step 7, divide homogenous region pixel subspace.
According to the following steps, using the homogenous region dividing method based on multinomial logistic regression prior model, to homogeneous Area pixel subspace is split, and obtains the segmentation result of homogenous region pixel subspace:
Step 1 arbitrarily chooses a pixel, centered on selected pixel from the pixel subspace of homogenous region The square window for establishing 3 × 3 calculates the standard deviation sigma of the window1
The side length of square window is increased by 2, obtains new square window, calculate the standard deviation of new square window by step 2 σ2
Step 3, if standard deviation threshold method T3=3, if | σ12| < T3, then it is σ by standard deviation2Square window as most Whole square window executes step 4;Otherwise, step 2 is executed;
Step 4 calculates the prior probability of center pixel in square window according to the following formula:
Wherein, p1' indicating that the prior probability of center pixel in square window, η ' indicate probabilistic model parameter, η ' value is 1, xk" indicating the number of pixels for belonging to kth ' class in square window, k' ∈ [1 ..., K'], K' indicate the classification number of segmentation, K' Value is 5, xi' indicate the number of pixels for belonging to the i-th ' class in the obtained square window of step 3;
The probability density of pixel grey scale is multiplied with the probability density of texture, obtains likelihood probability p' by step 62, wherein The probability density of gray scale is distributed to obtain by fading channel Nakagami, and the probability density of texture is distributed to obtain by t;
Step 7, by prior probability p1' and likelihood probability p2' be multiplied, obtain posterior probability p12';
Step 8 judges whether there are also untreated pixels in the pixel subspace of homogenous region, if so, executing step 1; Otherwise, step 9 is executed;
Step 9 obtains the segmentation result of homogenous region pixel subspace according to maximum posteriori criterion.
Step 8, by mixing aggregated structure atural object pixel subspace, homogenous region pixel subspace and structure-pixel subspace Segmentation result merge, obtain the final segmentation result of synthetic aperture radar SAR image.
Effect of the invention is further described below with reference to analogous diagram.
1. simulated conditions:
The hardware condition that the present invention emulates are as follows: Intellisense and image understanding laboratory graphics workstation;Present invention emulation Used synthetic aperture radar SAR image are as follows: the Piperiver that Ku wave band resolution ratio is 1 meter schemes.
2. emulation content:
Emulation experiment of the invention is split to the Piperiver figure in SAR image, as shown in Fig. 2 (a) Piperiver figure.The synthetic aperture radar SAR image that the figure is 1 meter from Ku wave band resolution ratio.
Using SAR image sketch step of the invention, to retouching of Piperiver pixel shown in Fig. 2 (a), obtain as Sketch map shown in Fig. 2 (b).
Sketch map compartmentalization shown in Fig. 2 (b) is obtained such as Fig. 2 using division pixel subspace step of the invention (c) administrative division map shown in.White space in Fig. 2 (c) indicates aggregation zone, other for no sketch line region and structural area Domain.Administrative division map shown in Fig. 2 (c) is mapped to the figure of Piperiver shown in Fig. 2 (a), obtains the Piperiver as shown in Fig. 2 (d) Image blend aggregated structure atural object pixel subspace figure.Structural region in administrative division map shown in Fig. 2 (c) is mapped to shown in Fig. 2 (b) Sketch map obtains the corresponding sketch line of structural region shown in Fig. 3 (a).The corresponding sketch line of structural region shown in Fig. 3 (b) In, black is the sketch line for representing line target, and in the corresponding sketch line of structural region shown in Fig. 3 (c), black is that representative is independent The sketch line of target.
Aggregated structure atural object pixel subspace step is mixed using segmentation SAR image of the invention, to shown in Fig. 2 (d) The mixing aggregated structure atural object pixel subspace figure of Piperiver figure is split, and obtains mixing aggregation knot shown in Fig. 4 (a) Structure atural object pixel subspace segmentation result figure, grey area indicate untreated ground object space, the area of remaining same color Domain representation same ground object space, the region of different colours indicate different ground object spaces.
Image Segmentation Methods Based on Features pinpoint target step is gathered based on sketch line using of the invention, is obtained independent shown in Fig. 4 (b) The segmentation result of target.
Using combination and segmentation result step of the invention, it is empty to merge mixing aggregated structure atural object pixel shown in Fig. 4 (a) Between segmentation result, homogenous region pixel subspace segmentation result and structure-pixel subspace segmentation result, obtain Fig. 4 (c), Fig. 4 (c) be Fig. 2 (a) Piperiver image final segmentation result figure, wherein black line-like area indicate line target segmentation knot Fruit.
3. simulated effect is analyzed:
Fig. 4 (c) is final segmentation result figure of the method for the present invention to Piperiver image, and Fig. 4 (d) is regarded based on level Feel semantic and the hidden model of adaptive neighborhood multinomial SAR image segmentation method to the final segmentation result of Piperiver image Figure can show that the method for the present invention determines the boundary of mixing aggregated structure atural object pixel subspace by comparing segmentation result figure More accurate, for homogenous region pixel subspace segmentation, classification number is more reasonable, and region consistency is obviously preferable, and right Pinpoint target in structure-pixel subspace has carried out preferable dividing processing.Using the method for the present invention to synthetic aperture radar SAR image is split, and is obtained the good segmentation effect of SAR image, be can be used for the semantic segmentation of SAR image.

Claims (10)

1. a kind of stochastic gradient Bayes's SAR image segmentation method based on sketch structure, includes the following steps:
(1) SAR image sketch:
(1a) inputs synthetic aperture radar SAR image;
(1b) establishes the sketch model of synthetic aperture radar SAR image;
The sketch map of (1c) from sketch model extraction synthetic aperture radar SAR image;
(2) pixel subspace is divided:
(2a) uses sketch line fields method, carries out compartmentalization processing to the sketch map of synthetic aperture radar SAR image, obtains The administrative division map of synthetic aperture radar SAR image including aggregation zone, without sketch line region and structural region;
(2b) will include aggregation zone, the administrative division map without sketch line region and structural region, be mapped to synthetic aperture radar SAR figure As in, obtain the mixing aggregated structure atural object pixel subspace of synthetic aperture radar SAR image, homogenous region pixel subspace and Structure-pixel subspace;
(3) stochastic gradient variation Bayesian network model is constructed:
(3a) according to the following formula, calculates the input layer of stochastic gradient variation Bayesian network model to the intermediate variable of hidden layer:
Wherein, hφIndicate stochastic gradient variation Bayesian network model input layer to hidden layer intermediate variable,Indicate with The input layer of machine gradient variation Bayesian network model is to intermediate variable hφConnection weight, m indicate hidden layer neuron number, The neuron number of m=500, n expression input layer, n=441,It indicatesCorresponding bias vector;
(3b) according to the following formula, calculates the approximate posterior probability of stochastic gradient variation Bayesian network model:
Wherein, qφ(z | x) indicates the approximate posterior probability of stochastic gradient variation Bayesian network model,It indicates Mean vector is μφ, covariance matrix isNormal distribution,Indicate stochastic gradient variation Bayesian network model Intermediate variable h of the input layer to hidden layerφWith μφConnection weight,It indicatesCorresponding bias vector,Indicate defeated Enter layer to hidden layer intermediate variable hφWith σφConnection weight,It indicatesCorresponding bias vector;
(3c) according to the following formula, calculates the hidden layer of stochastic gradient variation Bayesian network model to the intermediate variable of reconstruction of layer:
Wherein, hθIndicate stochastic gradient variation Bayesian network model hidden layer to reconstruction of layer intermediate variable,Hidden layer arrives Intermediate variable hθConnection weight,It indicatesCorresponding bias vector;
(3d) according to the following formula, calculates the conditional probability of stochastic gradient variation Bayesian network model:
Wherein, pθ(y | z) indicates the conditional probability of stochastic gradient variation Bayesian network model,Indicate mean value Vector is μθ, covariance matrix isNormal distribution,Intermediate variable h of the expression hidden layer to reconstruction of layerθWith μθCompany Weight is connect,It indicatesCorresponding bias vector,Intermediate variable h of the expression hidden layer to reconstruction of layerθWith σθConnection weight Value,It indicatesCorresponding bias vector;
(3e) according to the following formula, calculates the variation lower bound of stochastic gradient variation Bayesian network model:
Wherein, J (θ, φ) indicates that the variation lower bound of stochastic gradient variation Bayesian network model, φ indicate stochastic gradient variation shellfish The variational parameter of this network model of leaf,θ indicates stochastic gradient variation Bayes The generation parameter of network model,DKL(qφ(z|x)||pθ(z)) q is indicatedφ(z| And p x)θ(z) relative entropy between, z indicate the hidden layer variable of stochastic gradient variation Bayesian network model, pθ(z) hidden layer is indicated The prior probability of variable z, ∑ () indicate sum operation, and L indicates that hidden layer variable z carries out the number of Gauss sampling, log () table Show log operations, zlIndicate that value is by formula to the l times Gauss sampled result of zIt obtains, wherein ⊙ indicates point multiplication operation, εlIndicate that Gauss samples auxiliary variable, εl~N (0, I) indicates that Gauss samples auxiliary variable and meets standard Normal distribution;
(3f) according to the following formula, calculates the structural remodeling error of stochastic gradient variation Bayesian network model:
Wherein, G (θ, φ) indicates the structural remodeling error of stochastic gradient variation Bayesian network model, and K indicates input picture block Sum, xiIndicate i-th of input picture block, yiIndicate xiReconstructed image block, SM () expression ask sketch block operate, C () Expression asks sketch line length in sketch block to operate;
(4) feature learning is carried out to mixing aggregated structure atural object pixel subspace:
(4a) to the mixing aggregated structure atural object pixel subspace of synthetic aperture radar SAR image, spatially on connectivity into Row region division executes (6) if only existing a mutual not connected region;
(4b) to each mutual not connected region is carried out that it is corresponding multiple to obtain each region every a sampling by 21 × 21 window Image block sample;
(4c) takes and the one-to-one sketch block sample of image block sample each image block sample in sketch map;
(4d) produces corresponding one group of each region and meets uneven atural object distribution G to each mutual not connected region0Distribution Random number;
(4e) to each mutually not connected region, with the corresponding one group of random number in each region to stochastic gradient variation Bayesian network Connection weight initialized, the stochastic gradient variation Bayesian network after being initialized;
(4f) to it is each mutually not connected region initialization after the variation Bayesian network of gradient immediately, using image block sample as with The input layer of machine gradient variation Bayesian network, using the method for the stochastic gradient variation Bayesian inference of sketch structural constraint, Structural constraint training is carried out to the stochastic gradient variation Bayesian network after initialization, the stochastic gradient variation shellfish after being trained This network of leaf;
(4g) is to each mutual not connected region, the weight of the stochastic gradient variation Bayesian network after taking it to train, as the area The characteristic set in domain;
(5) segmentation SAR image mixes aggregated structure atural object pixel subspace:
(5a) splices the characteristic set of all mutual connected regions, using spliced characteristic set as code book;
(5b) calculates separately the inner product with each feature in code book to all features of each mutual not connected region, obtains every Projection vector of all features in a region on code book;
(5c) it is one corresponding to obtain each region to each mutually all projection vectors progress maximum value convergence of connected region Structural eigenvector;
(5d) propagates AP clustering algorithm using neighbour, does not cluster, obtains to the structural eigenvector of all mutual connected regions Mix the segmentation result of aggregated structure atural object pixel subspace;
(6) segmenting structure pixel subspace:
(6a) uses vision semantic rules, divides line target;
The feature of gathering of (6b) based on sketch line divides pinpoint target;
The result that (6c) divides line target and pinpoint target merges, and obtains the segmentation result of structure-pixel subspace;
(7) divide homogenous region pixel subspace:
Using the homogenous region dividing method based on multinomial logistic regression prior model, homogenous region pixel subspace is carried out Segmentation, obtains the segmentation result of homogenous region pixel subspace;
(8) combination and segmentation result:
The segmentation result of aggregated structure atural object pixel subspace, homogenous region pixel subspace and structure-pixel subspace will be mixed Merge, obtains the final segmentation result of synthetic aperture radar SAR image.
2. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In establishing the sketch model of synthetic aperture radar SAR image described in step (1b), specific step is as follows:
Step 1 arbitrarily chooses a number, the sum as template in [100,150] range;
Step 2 constructs a template on the side being made of pixel with different directions and scale, line, utilizes the side of template To with dimensional information structural anisotropy's Gaussian function, by the Gaussian function, the weighting system of each pixel in calculation template It counts, the weighting coefficient of all pixels point in statistical mask, wherein scale number value is 3~5, and direction number value is 18;
Step 3, according to the following formula, pixel is equal in calculating synthetic aperture radar SAR image corresponding with template area coordinate Value:
Wherein, μ indicates the mean value of all pixels point in synthetic aperture radar SAR image corresponding with template area coordinate, ∑ Indicate sum operation, g indicates the corresponding coordinate of any one pixel in the Ω region of template, and ∈ expression belongs to symbol, wg Indicate weight coefficient of the pixel at coordinate g in the Ω region of template, wgValue range be wg∈ [0,1], AgIndicate with The value of pixel of the pixel at the coordinate g in corresponding synthetic aperture radar SAR image in the Ω region of template;
Step 4 calculates the side of pixel in synthetic aperture radar SAR image corresponding with template area coordinate according to the following formula Difference:
Wherein, ν indicates the variance yields of all pixels point in synthetic aperture radar SAR image corresponding with template area coordinate;
Step 5 calculates the response that each pixel in synthetic aperture radar SAR image is directed to ratio operator according to the following formula:
Wherein, R indicates response of each pixel for ratio operator, min { } expression in synthetic aperture radar SAR image Minimum Value Operations, a and b respectively indicate two different regions in template, μaAll pixels point is equal in expression template area a Value, μbIndicate the mean value of all pixels point in the b of template area;
Step 6 calculates the response that each pixel in synthetic aperture radar SAR image is directed to correlation operator according to the following formula:
Wherein, C indicate synthetic aperture radar SAR image in each pixel be directed to correlation operator response,Expression square Root operation, a and b respectively indicate two different zones, ν in templateaIndicate the variance yields of all pixels point in a of template area, νbTable Show the variance yields of all pixels point in the b of template area, μaIndicate the mean value of all pixels point in a of template area, μbIndicate template region The mean value of all pixels point in the b of domain;
Step 7 calculates the response that each pixel in synthetic aperture radar SAR image is directed to each template according to the following formula:
Wherein, F indicate synthetic aperture radar SAR image in each pixel be directed to each template response,Expression square Root operation, R and C respectively indicate pixel in synthetic aperture radar SAR image to scheme for ratio operator and synthetic aperture radar SAR Pixel is directed to the response of correlation operator as in;
Step 8, judges whether constructed template is equal to the sum of selected template, if so, executing step 2, otherwise, executes Step 9;
Step 9, selection has the template of maximum response from each template, as the template of synthetic aperture radar SAR image, And using the maximum response of the template as the intensity of pixel in synthetic aperture radar SAR image, the direction of the template is made For the direction of pixel in synthetic aperture radar SAR image, the sideline response diagram and gradient of synthetic aperture radar SAR image are obtained Figure;
Step 10 calculates the intensity value of synthetic aperture radar SAR image intensity map, obtains intensity map according to the following formula:
Wherein, I indicates that the intensity value of synthetic aperture radar SAR image intensity map, r indicate synthetic aperture radar SAR image sideline Value in response diagram, t indicate the value in synthetic aperture radar SAR image gradient map;
Step 11 detects intensity map using non-maxima suppression method, obtains suggestion sketch;
Step 12 chooses the pixel suggested in sketch with maximum intensity, will suggest the pixel in sketch with the maximum intensity The pixel of point connection connects to form suggestion line segment, obtains suggestion sketch map;
Step 13 calculates the code length gain for suggesting sketch line in sketch map according to the following formula:
Wherein, CLG indicates to suggest the code length gain of sketch line in sketch map, and ∑ indicates sum operation, and J indicates current sketch The number of pixel, A in line neighborhoodjIndicate the observation of j-th of pixel in current sketch line neighborhood, Aj,0It indicates current In the case that sketch line cannot indicate structural information, the estimated value of j-th of pixel in the sketch line neighborhood, ln () indicate with E is the log operations at bottom, Aj,1Indicate the jth in the sketch line neighborhood in the case where current sketch line can indicate structural information The estimated value of a pixel;
Step 14 arbitrarily chooses a number, as threshold value T in [5,50] range;
Step 15 selects the suggestion sketch line of CLG > T in all suggestion sketch lines, is combined into synthetic aperture radar SAR figure The sketch map of picture.
3. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In specific step is as follows for sketch line fields method described in step (2a):
Sketch line is divided into table according to the concentration class of sketch line segment in the sketch map of synthetic aperture radar SAR image by step 1 Show the aggregation sketch line of aggregation atural object and indicates boundary, line target, the boundary sketch line of isolated target, line target sketch line, orphan Vertical target sketch line;
Step 2 chooses the sketch line segment work that concentration class is equal to optimal concentration class according to the statistics with histogram of sketch line segment concentration class For seed line-segment sets { Ek, k=1,2 ..., m }, wherein EkIndicate that any bar sketch line segment in seed line-segment sets, k indicate kind Sub-line section concentrates the label of any bar sketch line segment, and m indicates the total number of seed line segment, and { } indicates set operation;
Step 3, as basic point with the unselected line segment for being added to seed line-segment sets sum, with this basic point recursive resolve line-segment sets It closes;
Step 4 constructs the round primitive that a radius is the optimal concentration class section upper bound, with the circle primitive in line segment aggregate Line segment expanded, the line segment aggregate ecto-entad after expansion is corroded, is obtained in sketch map with sketch point being single The aggregation zone of position;
Step 5, to the sketch line for indicating boundary, line target and isolated target, during each sketch point with each sketch line is The heart constructs the geometry window that size is 5 × 5, obtains structural region;
Step 6 will remove the part other than aggregation zone and structural region as can not sketch region in sketch map;
Step 7, by sketch map aggregation zone, can not sketch region and structural region merge, obtain including aggregation zone, nothing The administrative division map of the synthetic aperture radar SAR image of sketch line region and structural region.
4. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In producing one group and meet uneven atural object distribution G to each mutually not connected region described in step (4d)0Distribution with Machine number, the specific steps of which are as follows:
Step 1 calculates the uneven atural object of synthetic aperture radar SAR image and is distributed G according to the following formula0The probability density of distribution:
Wherein, P (I (x, y)) indicates the probability density of the uneven atural object distribution of synthetic aperture radar SAR image, I (x, y) table Show that coordinate is the intensity value of the pixel of (x, y), n indicates the equivalent number of synthetic aperture radar SAR image, and α indicates synthesis hole The form parameter of diameter radar SAR image, γ indicate that the scale parameter of synthetic aperture radar SAR image, Γ () indicate gamma letter Number, value are obtained by following formula:
Wherein, u indicates that independent variable, ∫ indicate integration operation, and t indicates integration variable;
Step 2, to mixing aggregated structure atural object pixel subspace region Ri, using the intensity value of all pixels point in the region, adopt With the method for parameter estimation of Mellin transform, the uneven atural object distribution G of synthetic aperture radar SAR image is obtained0Needed for distribution Three parameter alphas, the estimated value of γ, n;
Step 3 randomly selects with mixing aggregated structure object space Ri500 image block samples, composition 500 × 441 matrix A;
Step 4 is distributed G by the uneven atural object of synthetic aperture radar SAR image using matrix B0The probability density letter of distribution Number generates one 500 × 441 matrix B, and the data in matrix B meet the uneven atural object point of synthetic aperture radar SAR image Cloth G0Distribution.
5. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In to each mutual not connected region described in step (4e), with the corresponding one group of random number in each region to stochastic gradient change Dividing the connection weight of Bayesian network to be initialized, specific step is as follows:
Step 1, using matrix B as the input layer x of stochastic gradient variation Bayesian network model to intermediate variable hφConnection weight Value
Step 2 randomly selects 500 column, the Matrix C of composition 500 × 500, using Matrix C as stochastic gradient variation from matrix B The intermediate variable h of Bayesian network modelφTo μφConnection weightUsing Matrix C as stochastic gradient variation Bayesian network The intermediate variable h of network modelφTo σφConnection weightUsing Matrix C as the hidden of stochastic gradient variation Bayesian network model Layer z to intermediate variable hθConnection weight
Step 3, using the transposition of matrix B as the intermediate variable h of stochastic gradient variation Bayesian network modelθTo μθConnection weight ValueUsing the transposition of matrix B as the intermediate variable h of stochastic gradient variation Bayesian network modelθTo μθConnection weight
6. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In specific step is as follows for the stochastic gradient variation Bayesian inference method of sketch structural constraint described in step (4f):
The prior probability of stochastic gradient variation Bayesian network model hidden layer is initialized as standardized normal distribution probability by step 1, The approximate posterior probability of stochastic gradient variation Bayesian network model is initialized as normal distribution probability, obtains stochastic gradient change Divide the analytic expression of the variation lower bound of Bayesian network model as follows:
Step 2 updates the generation parameter of stochastic gradient variation Bayesian network model according to the following formula:
Wherein, θt+1Indicate the generation parameter of stochastic gradient variation Bayesian network model after the t+1 times iteration, θtIt indicates the t times The generation parameter of stochastic gradient variation Bayesian network model after iteration,Expression asks inclined to the parameter θ of J (θ, φ) The operation led;
Step 3 updates the variational parameter of stochastic gradient variation Bayesian network model according to the following formula:
Wherein, φt+1Indicate the variational parameter of stochastic gradient variation Bayesian network model after the t+1 times iteration, φtIndicate t The variational parameter of stochastic gradient variation Bayesian network model after secondary iteration,Indicate the parameter φ to J (θ, φ) Ask the operation of local derviation;
Step 4 calculates structural remodeling error using the structural remodeling error formula of stochastic gradient variation Bayesian network model;
Step 5, judges whether structural remodeling error is less than threshold value 0.2, if so, executing step 5;Otherwise, step 1 is executed;
Step 6 completes the structural constraint training of stochastic gradient variation Bayesian network.
7. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In vision semantic rules described in step (6a) are as follows:
If i-th sketch line liWith j-th strip sketch line ljThe distance between be Dij, liDirection be Oi, ljDirection be Oj, i, j ∈ [1,2 ..., S], S are the total number of sketch line;
Width is greater than the line target of 3 pixels with two sketch line liAnd ljIt indicates, liAnd ljThe distance between DijLess than T1And Direction difference (Oi- Oj) less than 10 degree, wherein T1=5;
If the s articles sketch line lsGeometry window wsThe average gray of interior each column is AiIf the gray scale difference of adjacent column is ADi= |Ai-Ai+1|, if zs=[zs1,zs2,...,zs9] be adjacent column gray scale difference ADiLabel vector;
By width less than the line target of 3 pixels with single sketch line lsIt indicates, lsGeometry window wsIt is interior, calculate adjacent column Gray scale difference ADiIf ADi>T2, then zsi=1;Otherwise zsi=0, zsIn there are two element value be 1, remaining is 0, wherein T2 =34;
If L1,L2It is the set for indicating the sketch line of line target, if Dij<T1And | Oi- Oj| < 10, then li, lj∈L1;If sum (zs) =2, then ls∈L2, wherein sum () indicate to vector important summation operation.
8. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In specific step is as follows for segmentation line target described in step (6a):
Step 1, in structure-pixel subspace, according to the set L of the sketch line of line target1, by liAnd ljBetween region as line Target;
Step 2, in structure-pixel subspace, according to the set L of the sketch line of line target2, by lsRegion as line target.
9. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature exist In specific step is as follows for segmentation pinpoint target described in step (6b):
Step 1 would not indicate all sketch wire tags of line target in the structural region of administrative division map as candidate sketch line set In sketch line;
Step 2 randomly selects a sketch line from candidate sketch line set, during an endpoint with selected sketch line is The heart, the geometry window that construction size is 5 × 5;
Step 3 judges the endpoint that whether there is other sketch lines in geometry window, and if it exists, execute step 4;Otherwise, it executes Step 6;
Step 4 judges whether to only exist an endpoint, if so, sketch line where the endpoint and current sketch line are attached; Otherwise, step 5 is executed;
Step 5 connects the sketch line where selected sketch line and each endpoint, and it is maximum that wherein angle is chosen from all connecting lines Two sketch lines as connection complete sketch line;
Step 6 judges the endpoint that whether there is other sketch lines in the geometry window of another endpoint of sketch line, if depositing Executing step 4;Otherwise, step 7 is executed;
Step 7 chooses the sketch line comprising two and two or more sketch line segments, statistics to the sketch line for completing attended operation It include the item number n of sketch line segment, wherein n >=2 in selected sketch line;
Step 8, judges whether the item number n of sketch line is equal to 2, if so, executing step 9;Otherwise, step 10 is executed;
Step 9, by sketch line of the angle value on sketch line vertex in the range of [10 °, 140 °] as have gather feature Sketch line;
Step 10 selects sketch line of the angle value on the corresponding n-1 vertex of sketch line all in [10 °, 140 °] range;
Step 11 is defined as follows two kinds of situations in selected sketch line:
Whether the first situation judges adjacent (i-1)-th, the two sketch line segments of i-th sketch line segment, i+1 item in i-th element The same side of straight line, 2≤i≤n-1, if all sketch line segments and adjacent segments on sketch line are all same where retouching line segment Side, then marking the sketch line is with the sketch line for gathering feature;
Whether second situation judges adjacent (i-1)-th, the two sketch line segments of i-th sketch line segment, i+1 item in i-th element The same side of straight line where line segment, 2≤i≤n-1 are retouched, if having n-1 sketch line segment and adjacent segments on sketch line in the same side, And have a sketch line segment line segment adjacent thereto in non-the same side, also marking the sketch line is with the sketch line for gathering feature;
Step 12, an optional sketch line in having the sketch line for gathering feature are sat by two endpoints of selected sketch line Mark, determines the distance between two endpoints, if the end-point distances are in [0,20] range, then only using selected sketch line as expression The sketch line of vertical target;
Step 13, judge it is untreated have gather the sketch line of feature and whether all selected, if so, executing step 12;Otherwise, Execute step 14;
Step 14, with the method for super-pixel segmentation, around the sketch line for indicating pinpoint target in synthetic aperture radar SAR image Pixel carry out super-pixel segmentation, by super-pixel conduct of the gray value of super-pixel after segmentation in [0,45] or [180,255] Pinpoint target super-pixel;
Step 15 merges pinpoint target super-pixel, using the boundary of the pinpoint target super-pixel after merging as the side of pinpoint target Boundary obtains the segmentation result of pinpoint target.
10. stochastic gradient Bayes's SAR image segmentation method according to claim 1 based on sketch structure, feature It is, specific step is as follows for the homogenous region dividing method based on multinomial logistic regression prior model described in step (7):
Step 1 is arbitrarily chosen a pixel from the pixel subspace of homogenous region, is established centered on selected pixel 3 × 3 square window calculates the standard deviation sigma of the window1
The side length of square window is increased by 2, obtains new square window, calculate the standard deviation sigma of new square window by step 22
Step 3, if standard deviation threshold method T3=3, if | σ12| < T3, then it is σ by standard deviation2Square window as final Square window executes step 4;Otherwise, step 2 is executed;
Step 4 calculates the prior probability of center pixel in square window according to the following formula:
Wherein, p '1Indicate that the prior probability of center pixel in square window, η ' indicate that probabilistic model parameter, η ' value are 1, xk′′ Indicate the number of pixels for belonging to kth ' class in square window, k' ∈ [1 ..., K'], K' indicate the classification number of segmentation, and K' value is 5, xi' indicate the number of pixels for belonging to the i-th ' class in the obtained square window of step 3;
The probability density of pixel grey scale is multiplied with the probability density of texture, obtains likelihood probability p ' by step 52, wherein gray scale Probability density is distributed to obtain by fading channel Nakagami, and the probability density of texture is distributed to obtain by t;
Step 6, by prior probability p1' and likelihood probability p2' be multiplied, obtain posterior probability p12';
Step 7 judges whether there are also untreated pixels in the pixel subspace of homogenous region, if so, executing step 1;Otherwise, Execute step 8;
Step 8 obtains the segmentation result of homogenous region pixel subspace according to maximum posteriori criterion.
CN201611262026.9A 2016-12-30 2016-12-30 Stochastic gradient Bayes's SAR image segmentation method based on sketch structure Active CN106611422B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611262026.9A CN106611422B (en) 2016-12-30 2016-12-30 Stochastic gradient Bayes's SAR image segmentation method based on sketch structure

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611262026.9A CN106611422B (en) 2016-12-30 2016-12-30 Stochastic gradient Bayes's SAR image segmentation method based on sketch structure

Publications (2)

Publication Number Publication Date
CN106611422A CN106611422A (en) 2017-05-03
CN106611422B true CN106611422B (en) 2019-06-21

Family

ID=58636844

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611262026.9A Active CN106611422B (en) 2016-12-30 2016-12-30 Stochastic gradient Bayes's SAR image segmentation method based on sketch structure

Country Status (1)

Country Link
CN (1) CN106611422B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108961316B (en) * 2017-05-23 2022-05-31 华为技术有限公司 Image processing method and device and server
CN107403434B (en) * 2017-07-28 2019-08-06 西安电子科技大学 SAR image semantic segmentation method based on two-phase analyzing method
CN107464247B (en) * 2017-08-16 2021-09-21 西安电子科技大学 Based on G0Distributed random gradient variational Bayesian SAR image segmentation method
CN109165653B (en) * 2018-08-15 2022-03-15 西安电子科技大学 Extraction method of SAR image aggregation area based on semantic line segment neighbor connection
CN109190684B (en) * 2018-08-15 2022-03-04 西安电子科技大学 SAR image sample generation method based on sketch and structure generation countermeasure network
CN109344837B (en) * 2018-10-22 2022-03-04 西安电子科技大学 SAR image semantic segmentation method based on deep convolutional network and weak supervised learning
CN109816665B (en) * 2018-12-30 2020-10-16 苏州大学 Rapid segmentation method and device for optical coherence tomography image

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103745472A (en) * 2014-01-15 2014-04-23 西安电子科技大学 SAR (synthetic aperture radar) image segmentation method based on condition triplet Markov field
CN104036491A (en) * 2014-05-14 2014-09-10 西安电子科技大学 SAR image segmentation method based on area division and self-adaptive polynomial implicit model
CN104463882A (en) * 2014-12-15 2015-03-25 西安电子科技大学 SAR image segmentation method based on shape completion area chart and feature coding
CN105427313A (en) * 2015-11-23 2016-03-23 西安电子科技大学 Deconvolutional network and adaptive inference network based SAR image segmentation method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103745472A (en) * 2014-01-15 2014-04-23 西安电子科技大学 SAR (synthetic aperture radar) image segmentation method based on condition triplet Markov field
CN104036491A (en) * 2014-05-14 2014-09-10 西安电子科技大学 SAR image segmentation method based on area division and self-adaptive polynomial implicit model
CN104463882A (en) * 2014-12-15 2015-03-25 西安电子科技大学 SAR image segmentation method based on shape completion area chart and feature coding
CN105427313A (en) * 2015-11-23 2016-03-23 西安电子科技大学 Deconvolutional network and adaptive inference network based SAR image segmentation method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
贝叶斯集成框架下的极化SAR图像分类;陈博等;《西安电子科技大学学报(自然科学版)》;20150430;第42卷(第2期);第45-51页

Also Published As

Publication number Publication date
CN106611422A (en) 2017-05-03

Similar Documents

Publication Publication Date Title
CN106611422B (en) Stochastic gradient Bayes&#39;s SAR image segmentation method based on sketch structure
CN106611420B (en) The SAR image segmentation method constrained based on deconvolution network and sketch map direction
CN106651884B (en) Mean field variation Bayes&#39;s SAR image segmentation method based on sketch structure
CN106683102B (en) SAR image segmentation method based on ridge ripple filter and convolutional coding structure learning model
CN106611423B (en) SAR image segmentation method based on ridge ripple filter and deconvolution structural model
CN108549891B (en) Multi-scale diffusion well-marked target detection method based on background Yu target priori
Zhang et al. Boundary-constrained multi-scale segmentation method for remote sensing images
CN106846322B (en) The SAR image segmentation method learnt based on curve wave filter and convolutional coding structure
CN106611421B (en) The SAR image segmentation method constrained based on feature learning and sketch line segment
CN106909902B (en) Remote sensing target detection method based on improved hierarchical significant model
CN107403434B (en) SAR image semantic segmentation method based on two-phase analyzing method
CN102982338B (en) Classification of Polarimetric SAR Image method based on spectral clustering
Liu et al. Hierarchical semantic model and scattering mechanism based PolSAR image classification
Arbelaez et al. Constrained image segmentation from hierarchical boundaries
CN105930815A (en) Underwater organism detection method and system
CN104408731B (en) Region graph and statistic similarity coding-based SAR (synthetic aperture radar) image segmentation method
CN109829519B (en) Remote sensing image classification method and system based on self-adaptive spatial information
CN109446894A (en) The multispectral image change detecting method clustered based on probabilistic segmentation and Gaussian Mixture
CN107464247B (en) Based on G0Distributed random gradient variational Bayesian SAR image segmentation method
CN109886146A (en) Flood information remote-sensing intelligent acquisition method and equipment based on Machine Vision Detection
CN112767413A (en) Remote sensing image depth semantic segmentation method integrating region communication and symbiotic knowledge constraints
CN107292268A (en) The SAR image semantic segmentation method of quick ridge ripple deconvolution Structure learning model
CN105512670A (en) HRCT peripheral nerve cutting based on KECA feature dimension reduction and clustering
CN110136143A (en) Geneva based on ADMM algorithm multiresolution remote sensing image segmentation method off field
Albert et al. Land use classification using conditional random fields for the verification of geospatial databases

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant