CN109559321A - A kind of sonar image dividing method and equipment - Google Patents
A kind of sonar image dividing method and equipment Download PDFInfo
- Publication number
- CN109559321A CN109559321A CN201811435393.3A CN201811435393A CN109559321A CN 109559321 A CN109559321 A CN 109559321A CN 201811435393 A CN201811435393 A CN 201811435393A CN 109559321 A CN109559321 A CN 109559321A
- Authority
- CN
- China
- Prior art keywords
- tree
- spanning tree
- directed graph
- sonar
- max
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/174—Segmentation; Edge detection involving the use of two or more images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Abstract
The invention discloses a kind of sonar image dividing method and equipment, which comprises establishes non-directed graph using sonar initial data;The non-directed graph is split by minimum spanning tree and maximum spanning tree simultaneously in the non-directed graph, wherein, background area is marked off in the non-directed graph by the minimum spanning tree, target area is marked off in the non-directed graph by the maximum spanning tree.The embodiment of the present invention uses sonar initial data in cutting procedure, does not lose the information of target maximumlly, reduces influence of the noise for cutting procedure;The it is proposed of the embodiment of the present invention is simultaneously split the non-directed graph by minimum spanning tree and maximum spanning tree in the non-directed graph, it has taken into account sonar datum target field gradient and has fluctuated big, the unconspicuous feature of background area Fluctuation of gradient, obtained the accurate segmentation effect for target detection.
Description
Technical field
The present invention relates to field of image processing, espespecially a kind of sonar image dividing method and equipment.
Background technique
It is explored with the development of the technology of the mankind and to naturally continuous, it is more and more deep to the understanding of Yu Haiyang, it is various
Marine resources become the reserved resources to become more and more important.Frequent all kinds of underwater operations need good prospecting tools.Acoustic signal
Underwater decaying is low, has farther away visual range, underwater performance is far superior to other imaging modes, thus becomes water
The main tool of lower detection.Object detection field under water, since the bandwidth of subsurface communication is limited, power limited is swept by side
Real-time data transmission to computer and is carried out artificial detection not only time consumption and energy consumption by Sonar system, but also shape when due to Imaging sonar
At deformation and noise make image and optical imagery that there is relatively big difference, be not easy intuitivism apprehension, the artificial inspection to sonar target
It surveys, needs to detect operator with experience very rich, but actually detected effect is also bad.Therefore, to real-time sonar number
Divide detection and judgement automatically according to target is carried out, it will reduce underwater data and transmit pressure, accelerate the efficiency of target detection identification
And accuracy rate.
Sonar scans sub-sea bottom, and the sonar image containing target is generally divided into three parts: (1) target highlight regions;
(2) shadow region;(3) bottom reverberation region.Target highlight regions corresponding with shadow region are sound waves on object
The strong echo area reflected to form and object block so that the acoustics shadow region that rear can not receive sound wave and reflect to form.
Remaining region is so-called bottom reverberation region, caused by it is sound scattering as coarse seabed.Usual highlight regions
And the shapes and sizes of shadow region can be used for the classification and analysis of target object.But since bottom reverberation region has very
Strong speckle noise causes very big difficulty to the segmentation of sonar image.
It is the method by Threshold segmentation in early days for the dividing method of sonar image submarine target, according to mesh in image
The grey value difference of mark, shade and background extracts in target area, shadow region from the background area of seabed.This method is in image
Target differ larger with background gray levels and effect is preferable when target gray level is concentrated very much, on the contrary then effect is deteriorated.Due to
Imaging sonar feature, target gray level can with the distance of target range sonar increase and reduce, merely foundation threshold value come into
Row segmentation often makes segmentation result malfunction.Several new Underwater Image dividing methods were proposed in recent years, mainly included mould
Paste cluster segmentation method, the split plot design based on active contour, markov random file split plot design and the split plot design based on fractal theory
Deng.
Partitioning algorithm based on fuzzy clustering calculates every bit to the person in servitude of cluster centre after determining initial cluster center
Category degree, after according to subjection degree update cluster centre, until each sample be subordinate to angle value stablize.The algorithm introduces fuzzy
The concept of mathematics has preferable noise immunity, and segmentation precision is high, but update every time cluster centre Shi Douxu recalculate it is each
The degree of membership of a point to former cluster centre is divided low efficiency, is not suitable for the image segmentation of the real-time underwater sound with duration.Based on work
The split plot design of driving wheel exterior feature, key idea are to choose initial zero level set function, the thought minimized using region energy, by zero
Level set function drives the edge to target area, realizes segmentation.Wherein the selection of zero level set function is particularly significant, if selection
It is improper, erroneous segmentation can be caused because falling into local energy minima.Therefore such methods are usually as fine segmentation, rough
It is used after obtaining target shape, makes it possible to accurately extract object boundary.Method based on markov random file can be tied
Context information is closed, constantly segmentation result is improved.Each pixel value is regarded as with the random of certain probability distribution
Process establishes the Label Field for indicating segmentation result using this probability density function and Markov random field model, and looks for
It is combined to the object for obtaining image with maximum probability.This method can each pixel generic in accurate description image, with
And its dependence between surrounding pixel, if but want to realize accurate segmentation, need point of the clear pixel in each region
Cloth feature.Split plot design based on fractal theory is split using the Cancers Fractional Dimension Feature of image.Natural texture, which has, divides shape
Feature, and culture does not have fractal characteristic, therefore different from the fractal dimension of natural scenic spots by culture, it can be by mesh
Mark is split.
In addition, existing method is handled for sonar image, and when sonar data are converted into image, need by
A series of transformation, these transformation meetings can also enhance noise to mesh so that the image impairment strong reflectance signal of partial target
Target interference, leads to the accuracy divided on sonar image, Small object is be easy to cause to lose;If directly using existing
Method handles sonar initial data, due to the different characteristics of sonar data and sonar image, often will cause target
Erroneous segmentation.
Summary of the invention
In order to solve the above-mentioned technical problems, the present invention provides a kind of sonar image dividing method and equipment, make full use of
Sonar echo data feature has good tolerance to uniform ambient noise, and can obtain accurate segmentation result.
In order to reach the object of the invention, the present invention provides a kind of sonar image dividing methods, comprising:
Non-directed graph is established using sonar initial data;
The non-directed graph is split by minimum spanning tree and maximum spanning tree simultaneously in the non-directed graph,
In, background area is marked off in the non-directed graph by the minimum spanning tree, by the maximum spanning tree in the nothing
Target area is marked off into figure.
It is optionally, described to establish non-directed graph using sonar initial data, comprising:
Point all in the sonar initial data is constituted into point set V, by connection phases all in the sonar initial data
The collection E when constituting of adjacent two o'clock, the difference of the data value of adjacent two o'clock are the weight on side, point set V and side collection E constitute non-directed graph G (E,
V)。
Optionally, the non-directed graph is split by the minimum spanning tree, comprising:
The weight on side in the non-directed graph is sorted from small to large, successively the side in the non-directed graph is merged and is sentenced
It is disconnected, meet in two trees that two vertex on the side that currently grown are not belonging to same tree and described two vertex are adhered to separately
When minimum spanning tree similarity Condition, two trees of the side connection that currently grow are merged.
Optionally, two trees meet minimum spanning tree similarity Condition and include:
Dmin(C1, C2) < min (Dmin(C1)+d1, Dmin(C2)+d2)
Wherein,
Dmin(C1, C2) be the side that currently grown weight minimum value, min (Dmin(C1)+d1, Dmin(C2)+
d2) indicate to take Dmin(C1)+d1And Dmin(C2)+d2In minimum value, Dmin(C1) it is tree C1The weight maximum value on middle side, Dmin(C2)
To set C2The weight maximum value on middle side, size (C1) it is tree C1In number of nodes, size (C2) it is tree C2In number of nodes, ε is institute
State the mean value of all side right weights in non-directed graph.
Optionally, the non-directed graph is split by the maximum spanning tree, comprising:
The weight on side in the non-directed graph is sorted from large to small;
In maximum spanning tree initial growth, determine that the side that currently grown belongs to target area;
Same tree and adhere to separately two of described two vertex are not belonging on two vertex on the side that currently grown
When tree meets maximum spanning tree similarity Condition, two trees of the side connection that currently grow are merged.
Optionally, described in maximum spanning tree initial growth, determine that the side that currently grown belongs to target area,
Include:
In the maximum spanning tree initial growth, determine that the maximum value on the side vertex Zhong Liangge that currently grown is big
In preset threshold value.
Optionally, two trees meet maximum spanning tree similarity Condition and include:
Dmax(C1, C2) > max (Dmax(C1)-k1, Dmax(C2)-k2)
Wherein,
Dmax(C1, C2) be the side that currently grown weight maximum value, max (Dmax(C1)-k1, Dmax(C2)-
k2) indicate to take Dmax(C1)-k1And Dmax(C2)-k2In maximum value, Dmax(C1) it is tree C1The weight minimum value on middle side, Dmax(C2)
To set C2The weight minimum value on middle side, size (C1) it is tree C1In number of nodes, size (C2) it is tree C2In number of nodes,Indicate tree C1The weight limit on middle side and the difference of minimal weight,Indicate tree C2The weight limit on middle side and the difference of minimal weight.
Optionally, the minimum spanning tree and maximum spanning tree are grown simultaneously, the method also includes: in the most your pupil
In Cheng Shu and maximum spanning tree growth course, to the information real-time update of each tree, the content of update includes at least one following:
The quantity for the point for including in tree;
The boundary position information of tree;
The center position information of tree;
The class label information of tree.
Optionally, it is described in the non-directed graph simultaneously by minimum spanning tree and maximum spanning tree to the non-directed graph into
After row segmentation, further includes:
Merge overdivided region.
Optionally, the merging overdivided region, comprising:
The linear areas detected is closed according to the center in the center for obtaining each target area
And.
Optionally, after the merging overdivided region, the method also includes: size detection is carried out to target area
And/or conspicuousness detection, will test by target area as real goal, will test unsanctioned target area as void
Decoy.
The present invention also provides a kind of sonar image splitting equipments, comprising: memory, processor and storage are on a memory simultaneously
The computer program that can be run on a processor, the processor realize the sonar image segmentation side when executing described program
Method.
The embodiment of the present invention includes: non-directed graph is established using sonar initial data;Simultaneously by most in the non-directed graph
Small spanning tree and maximum spanning tree are split the non-directed graph, wherein by the minimum spanning tree in the non-directed graph
In mark off background area, target area is marked off in the non-directed graph by the maximum spanning tree.The embodiment of the present invention
Sonar initial data is used in cutting procedure, does not lose the information of target maximumlly, reduces noise for cutting procedure
It influences;The it is proposed of the embodiment of the present invention is in the non-directed graph simultaneously by minimum spanning tree and maximum spanning tree to described undirected
Figure is split (the two poles of the earth spanning forest algorithm), has taken into account sonar datum target field gradient and has fluctuated big, background area Fluctuation of gradient not
Obvious feature has obtained the accurate segmentation effect for target detection.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
It obtains it is clear that understand through the implementation of the invention.The objectives and other advantages of the invention can be by specification, right
Specifically noted structure is achieved and obtained in claim and attached drawing.
Detailed description of the invention
Attached drawing is used to provide to further understand technical solution of the present invention, and constitutes part of specification, with this
The embodiment of application technical solution for explaining the present invention together, does not constitute the limitation to technical solution of the present invention.
Fig. 1 is sonar initial data and image data comparison diagram;
Fig. 2 is sonar initial data and its gradient map correlation comparison diagram;
Fig. 3 is the flow chart of the sonar image dividing method of the embodiment of the present invention;
Fig. 4 is the sonar data graphs and Weibull matched curve figure of the embodiment of the present invention;
Fig. 5 is the sonar data background and clear zone probability density curve figure of the embodiment of the present invention;
Fig. 6 is the generation schematic diagram of the two poles of the earth spanning forest algorithm of the embodiment of the present invention;
Fig. 7 is the flow chart of the sonar image dividing method of another embodiment of the present invention;
Fig. 8 is the sonar image and its initial segmentation result exemplary diagram of the embodiment of the present invention;
Fig. 9 is the recognition methods to the curve for having slight curvature of the embodiment of the present invention;
Figure 10 is the sonar image and its final segmentation result exemplary diagram of the embodiment of the present invention;
Figure 11 is the structural block diagram of the detection method of the embodiment of the present invention;
Figure 12 is sonar image deformation schematic diagram;
Figure 13 is target area and background area in the conspicuousness detection of the embodiment of the present invention;
Figure 14 is the schematic diagram of the sonar image segmenting device of the embodiment of the present invention;
Figure 15 is the schematic diagram of the sonar image splitting equipment of the embodiment of the present invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to the present invention
Embodiment be described in detail.It should be noted that in the absence of conflict, in the embodiment and embodiment in the application
Feature can mutual any combination.
Step shown in the flowchart of the accompanying drawings can be in a computer system such as a set of computer executable instructions
It executes.Also, although logical order is shown in flow charts, and it in some cases, can be to be different from herein suitable
Sequence executes shown or described step.
The high resolution of sonar initial data, data volume is big, in display, needs by a series of transformation.Data first
Carry out it is down-sampled, by lateral resolution from more than 10,000 be reduced to adapt to screen show 2000 or so, while by length be 2 bytes
Data 1 byte length that suitable gray level image show is compressed to by the adjustment of window width and window level, finally progress gamma is converted pair
Image is enhanced, and the visual effect of gray level image is optimized.Fig. 1 is initial data and pair Jing Guo transformed image data
Than.As seen from Figure 1, in initial data after above-mentioned processing, the image impairment strong reflectance signal of target enhances noise pair
The interference of real goal, so that the accuracy divided on gray level image, is especially easy to lose point for Small object
It cuts.
Gradient analysis is carried out to initial data, defines the intensity difference that gradient is adjacent pixel values.Since data are not in 0-255
Between, it can not directly be indicated with image, therefore pseudo-colours dyeing is carried out to all data, data value range is 0-5000, value
It is more high warmer.As shown in Fig. 2, left side one is classified as the colored graph of initial data, the right one is classified as the dyeing of corresponding gradient map
Figure can find that sonar initial data and its gradient map have a very strong correlation to this, and each region inner gradient and region
Positive correlation is presented in reflection echo intensity, and the echo-signal inside region is stronger, and fluctuation is more significant inside region.Such data
If directly being handled with the method for image procossing, since sonar target zone luminance value is larger, internal fluctuation is significant, when
Using image procossing method when, this significant fluctuation is understood to be extremely strong noise so that segmentation effect be deteriorated, sternly
It will cause erroneous segmentations in weight situation.
In the embodiment of the present invention, according to above-mentioned sonar initial data feature and the feature of target area strong reflection, propose
The algorithm of two-stage spanning forest.
The two poles of the earth spanning forest algorithm is that maximum spanning tree is introduced in cutting procedure on the basis of minimum spanning tree, is made
In initial data weak reflecting background region by minimum spanning tree be divided into it is one or more integrally, and strong reflection target area
Domain is divided into one or more whole by maximum spanning tree.Minimum spanning tree and maximum spanning tree are grown simultaneously, Zhi Daosuo
Some points are all traversed, and growth terminates.Therefore referred to as the two poles of the earth spanning forest algorithm.
In the embodiment of the present invention, after basic segmentation is carried out to image, also the part of over-segmentation is post-processed: in number
Commonplace linear areas is easy excessively to be segmented into multiple independent subregions in cutting procedure in, in order to reduce false-alarm
Rate is also detected and is merged to linear regions.
As shown in figure 3, the sonar image dividing method of the embodiment of the present invention, comprising:
Step 101, non-directed graph is established using sonar initial data.
If each point of sonar initial data is defined as v, the side for connecting adjacent two o'clock is defined as e, adjacent two o'clock viAnd vj's
The weight w of side e is defined as the difference of this two o'clock data value, it may be assumed that
Point all in the sonar initial data is constituted into point set V, by connection phases all in the sonar initial data
The collection E, point set V and side collection E when constituting of adjacent two o'clock constitute non-directed graph G (E, V).
Step 102, the non-directed graph is carried out by minimum spanning tree and maximum spanning tree simultaneously in the non-directed graph
Segmentation, wherein background area is marked off in the non-directed graph by the minimum spanning tree, is existed by the maximum spanning tree
Target area is marked off in the non-directed graph.
In this step, all sides are ranked up according to weighted value, are executed between minimum spanning tree and maximum spanning tree
Growth merges, wherein the growth merging between minimum spanning tree and maximum spanning tree is independent from each other.
For minimum spanning tree, the weight on side in the non-directed graph is sorted from small to large, successively in the non-directed graph
Side merge judgement, be not belonging to same tree and described two vertex point on two vertex on the side that currently grown
When the two tree similitudes consistent (meeting minimum spanning tree similarity Condition) belonged to, this edge is merged, i.e., it will be described current
Two trees of the side connection grown merge.Otherwise continue to merge according to a line under the sequential selection sequenced and sentence
It is disconnected.
Similarly, for maximum spanning tree, growing strategy is similar to minimum spanning tree, and weighted value is according to descending
Sequence sequence, then successively selection side merge judgement.
In minimum spanning tree growth course, judges to the similitude set two, first have to determine this respectively
The inside homogeneity of two trees: homogeneity D inside minimum spanning tree C is definedminIt (C), is the maximum of all side right weight values in tree C
Value:
The homogeneity D of this two trees of its secondary determinationmin(C1, C2), to be belonging respectively to two tree C1、C2Adjacent edge (i.e.
The side that currently grown) weight minimum value:
Under normal circumstances, if two tree homogeneity meet following condition, two trees can merge:
Dmin(C1, C2) < min (Dmin(C1), Dmin(C2))
But in the merging process of minimum spanning tree, it is frequently encountered the growth retardation problem of tree: the power on all sides in tree
Weight values are both less than not connected side adjacent thereto, and minimum spanning tree can not grow.In order to solve this problem, it defines internal equal
One property measures d:
Wherein size () refers to tree interior joint number, and ε is normal amount.The selection of ε largely affects segmentation result
Fine degree.When ε is too small, most of background area can not start to grow, and cause over-segmentation, when ε is excessive, the whole big portion of figure
Divide and be connected to a region, leakage is caused to divide.In sonar data, background area proportion is huge, clear zone ratio very little,
To ensure that background area may be grown, ε takes the mean value of all side right weights in non-directed graph, i.e.,
Wherein E is the side collection that side right all in non-directed graph is reconstructed into, number when N is in collection E.When initial growth,
Homogeneity measurement d occupies leading position during the growth process, and as the size of one tree increases, the value of d is reduced rapidly, interregional
Similitude judgement can gradually occupy leading factor.
When two tree homogeneity meet following minimum spanning tree similarity Condition, two minimum spanning trees can merge:
Dmin(C1, C2) < min (Dmin(C1)+d1, Dmin(C2)+d2)
Wherein, min (Dmin(C1)+d1, Dmin(C2)+d2) indicate to take Dmin(C1)+d1And Dmin(C2)+d2In minimum value,
Dmin(C1) it is tree C1The weight maximum value on middle side, Dmin(C2) it is tree C2The weight maximum value on middle side, size (C1) it is tree C1In
Number of nodes, size (C2) it is tree C2In number of nodes.
And in maximum spanning tree growth course, in order to enable merging obtained region by maximal tree growth is that target is bright
Area, before growth, it is thus necessary to determine that the growth conditions of maximal tree carries out a preliminary judgement in maximal tree initial growth,
Whether judge to be grown is belonging to target clear zone (determination belongs to target area when currently being grown).According to sonar
Statistical distribution feature, selected threshold T determine two tops in the side that currently grown in maximum spanning tree initial growth
The maximum value of point is greater than preset threshold value T.
That is, in maximal tree initial growth, that is, size (C) < 3, ifMaximal tree starts to give birth to
It is long.
The judgement principle of threshold value T is as follows:
Sonar smooth bottom data graphs obey Weibull distribution, probability density function expression formula are as follows:
X is stochastic variable in formula, and a > 0 is form parameter, and c > 0 is dimensional parameters.
But in fact, it is flat site that seabed major part region, which can not can be regarded as, wherein usually also including target clear zone
And shadow region.In sonar data, seabed background area brightness value is lower, and occupies the overwhelming majority of data, and highlight bar is bright
Angle value is higher, and proportion is seldom, and shadow region brightness value 100 hereinafter, and proportion less than 1%.To comprising clear zone and
The sonar data graphs of shadow region carry out Weibull function fitting, as shown in figure 4, sonar data histogram curve is at unimodal
Distribution, peak value appear in seabed background area, and clear zone occurs in the long-tail region at histogram rear.And it is fitted obtained Weibull
The reason of curve fit solution at peak value is preferable, occurs deviation at peak value end, causes a deviation is the appearance due to clear zone,
The excessively high pixel number of brightness value increases, and causes in histogram that downward trend slows down at peak of curve end.At this point, by straight
The curve that side's figure fitting obtains can determine the number of background dot in data, it can determine background area proportion.
The statistical result obtained according to before, the brightness value of background area is low, and proportion is big, and the brightness value of target area is high,
Proportion is small, it is believed that it is special that the histogram curve in sonar data graphs at peak value meets the distribution of seabed background area
Point.By described before, Weibull distribution is obeyed in seabed background area therefore can by the histogram distribution feature of sonar data
To determine clear zone property of the histogram.
If initial data histogram probability curve is F (x), it is bent that obtained background area probability density is nearby fitted by peak value
Line is fb(x), background area proportion is α, and target and shadow region proportion are β, and the histogram curve in clear zone is fo
(x), then above-mentioned curve meets following rule:
F (x)=α fb(x)+βfo(x)
The histogram curve in clear zone is fo(x) it can indicate are as follows:
To sum up, the probability density curve of background and clear zone in sonar data can be determined respectively.As shown in figure 5, by pattra leaves
This principle can be calculated in the case where determination is not background or clear zone, and the posterior probability of various brightness values passes through later
Minimal error rate is calculated, background and clear zone can be distinguished.Its minimal error rate P (C) calculation formula is as follows:
Wherein P (c1), P (c2) respectively indicate background and clear zone ratio shared in totality;P(x|c1), P (x | c2) table
Show the point proportion that brightness value is x in background and clear zone respectively, i.e., before obtained background and clear zone probability density.
In P (C) the smallest situation, the x that acquiresiIt is assigned to threshold value T, the initial growth conditions as maximal tree.
As size (C) > 3, to continued growth, then the similitude for also needing to set two judges maximal tree
(judging whether to meet maximum spanning tree similarity Condition).First have to the inside homogeneity for determining this two trees respectively: definition
Homogeneity D inside maximum spanning tree Cmax(C), it is the minimum value of all side right weight values in tree C:
The homogeneity D of this two trees of its secondary determinationmax(C1, C2), to be belonging respectively to two tree C1、C2Adjacent edge (i.e.
The side that currently grown) weight maximum value:
Analyze target data it is found that target area center while weighted value be greater than target area boundaries position while
Weighted value, and to approach or be faster than linear decline by the weighted value fall off rate of center to boundary position side, define
Its internal homogeneity measurement i.e. fall off rate are k:
When two Pterostyraxs are when same strong reflection target area, it may appear that two kinds of situations: if the one, side that is connected is in target
The heart, then the weight on the side should be higher than that the side right weight values of object boundary;If the side that two, is connected is located at the boundary of target, weight
Value should meet target area intensity decline rule.Due to the position on side be always it is unknown, then all as situation two processing, be allowed to
Meet decline rule, determine that two maximum spanning trees can grow merging and should meet maximum spanning tree similarity Condition at this time:
Dmax(C1, C2) > max (Dmax(C1)-k1, Dmax(C2)-k2)
Wherein,
Dmax(C1, C2) it is to be belonging respectively to two tree C1And C2The weight maximum value on middle side, max (Dmax(C1)-k1, Dmax
(C2)-k2) indicate to take Dmax(C1)-k1And Dmax(C2)-k2In maximum value, Dmax(C1) it is tree C1The weight minimum value on middle side,
Dmax(C2) it is tree C2The weight minimum value on middle side, size (C1) it is tree C1In number of nodes, size (C2) it is tree C2In node
Number,Indicate tree C1The weight limit on middle side and the difference of minimal weight,Indicate tree C2The weight limit on middle side and the difference of minimal weight.
Fig. 6 simply describes the cutting procedure of the two poles of the earth spanning forest algorithm segmented image.
After maximum spanning tree and minimum spanning tree have grown a line simultaneously, the real time information to each tree is needed to carry out
It updates, comprising:
(1) dimension information: for recording the quantity for the point for being included in one tree.
(2) boundary position information: the extreme value of the upper and lower, left and right four direction in all coordinates of the tree.
(3) center position information: according to boundary position information, the center point coordinate up and down with left and right is taken, as the tree
The center position in region.
(4) class label information: each tree can be labeled to be grown by maximum or minimum spanning tree, as possible to it
It is the judgement of shade or target.
Different region is accidentally merged in order to prevent, in addition the big noise of sound spectrogram itself, point of above-mentioned design
Cutting the result of algorithm, there may be over-segmentations.As shown in fig. 7, after step 102, may also include that in the embodiment of the present invention
Step 103, merge overdivided region.
Wherein, the center for obtaining each target area, according to the center to the linear areas detected into
Row merges.
The information for each tree that above-mentioned segmentation obtains both other than it be used to control growth conditions, will be also used for next
The orientation of over-segmentation merges.
The landforms such as a large amount of existing terraced fields, ridge, gully, cables, can show the line of rule on sound spectrogram in seabed
Shape structure.The initial data of sonar is observed it can be found that echo strong reflection region is discontinuous in original sonar data.These
Linear structure is made of due to its biggish scale, presentation one section of one section of strong reflection echo area interval.Aforementioned maximum spanning tree
In growth course, the independent segment strong signal of each section can be split, very high false alarm rate be easy to cause, such as Fig. 8 institute
Show, wherein a, b, the Regional Representative that c is directed toward is divided into the region of different objects.In order to solve this problem, it needs to linear
Cut zone is detected and is merged.
During dividing before, the center in each region is recorded, passes through these later
Center carries out combining data detection to linear areas.All to connect straight central point, corresponding region is merged into together
One region.
Using each tree center position as the point (x in cartesian coordinate systemi, yi), with polar coordinate representation are as follows:
xi=ρ cos θ
yi=ρ sin θ
These points are mapped to hough space, obtain the sine curve equation that each pair of point is answered:
ρ=xicosθ+yisinθ
As shown in figure 9, find the center (a, b) of multiple spot aggregation, with (a, b) for the center of circle, in certain area of a circle it is all just
Chord curve is all identified as belonging to collinear point in cartesian coordinate system.It is above-mentioned due to straight lines rare in nature
Method can merge the linear areas for having slight curvature, as shown in Figure 10, completely illustrate the form of seabed terraced fields, while two
A suspected target region has also obtained correct segmentation.
Due to highlight regions such as some terraced fields in seabed, reefs, since Echo Rating is higher, it can equally pass through maximum spanning tree
It generates, so may include real goal and false target by the target area that step 102 and 103 obtain.On proving
The validity and accuracy for stating dividing method have carried out target detection, wherein right for the dividing method treated data
Target area carry out size detection and/or conspicuousness detection, will test by target area as real goal, will test not
By target area as false target.
As shown in figure 11, detection method can be based on two point features: target size and target conspicuousness.Target is in scale
Upper similitude with higher, it is available by the conversion relation to target size in target actual size and sonar image
One approximate range scale, by size detection, the suspicious region that area size can be obviously misfitted with target is rapid
It excludes.Target conspicuousness then refers to the target and the significant difference close to background.It is produced due to sonar multipath propagation etc.
Raw noise and distortion, and due to the water-bed texture of complexity that sea bottom complex environment generates, the cut zone in the region can be made
Quantity increases, to be easier to be detected false target.However the initial data in these regions usually have it is a wide range of, unstable
Strong reflection echo can will be complicated if the cut zone of each suspected target and surrounding environment compared
The object of erroneous segmentation excludes in environment.Detection method is as follows:
It is the region to be judged according to the degree of agreement of target area and realistic objective whether for the size detection of target
For real goal.By the length in clear zone, width in sound spectrogram, the actual length width of immersed body can be extrapolated.
And interference of the sonar data by boat posture, sonar image can be made to generate very big deformation, as shown in figure 12, this
A part of deformation includes three aspects: (1) the oblique rectangle generated due to the water-bed height of sonar distance is become;(2) by sonar edge
Y-axis rotate caused by 3 deformation of pitch angle and due to navigation turn to, sonar along X-axis turn to generate course angle α deformation;
(3) deformation for receiving the restriction of route speed v due to acquiring data and generating.Therefore, it is impossible to accurately sentence according to original sound spectrogram
It cuts off the water supply lower dimension of object size.But object deformation caused by boat posture can be eliminated by inverse transformation, to obtain
Close to the result of linear optics image.
In the detection process, the boat posture in terms of carrying out above three to target area first is corrected, according to correction
Target clear zone afterwards calculates the Length x Width of submarine target.If the two meets the requirement of real goal size, determine to be detected
Target area be real estate.
Conspicuousness detection for target, judges suspicious region by the significant difference between detection target and background
It whether is real goal.
At the two poles of the earth during spanning forest algorithm segmented image, a region can be because there are strength differences with peripheral region
And it is divided and comes out.Simultaneously as the multi-pathway effect and sea bottom complex environment of sonar, so that the initial data of background is usually
Also it will appear a wide range of, unstable strong reflection echo, due to the discontinuity of its brightness value and gradient value difference, dividing
Cheng Zhong, this part are often divided into several pieces of different zones, generate false target.These strong echo areas that segmentation is obtained
The initial data in domain carries out conspicuousness detection, it is found that the significant difference of itself and surrounding background area is not obvious, therefore, can be with
It is detected by conspicuousness and rejects part false target.
Segmentation result can provide the boundary position information of target area, using this information, can find calculating conspicuousness
Area size, as shown in figure 13, A1For target area, size is determined by the dimension information divided, long and wide difference
It is denoted as w1, h1;It apart from boundary is respectively w outside object boundary2, h2Region be A2。w2, h2The empirical regulation of size are as follows:
h1=h2, w1=w2
Target conspicuousness detection is carried out using Z detection algorithm, detection formula writing:
Wherein A1, A2Respectively indicate target area and background area, μ1、μ2、σ1、σ2Respectively target area and background area
Mean value and variance.When target background significant difference is big, Z detection function value increases, otherwise detection function minimum reachable 0,
Two region indifferences.
By the extraction to target size and significant characteristics, doubtful mesh can be carried out to the result divided before
Mark judgement.Size is carried out to segmented region and conspicuousness judges, if meeting the requirements, for real goal, exports target
Region, on the contrary it is used as false target, merge with background.
Finally, calculating recall rate R and false-alarm by the quantity for counting real goal and false target in all output results
Rate F.If TP is the real goal quantity detected, FN is the quantity of undetected real goal, and it is true that FP, which is by erroneous detection,
The false target of target.Then R and F can be expressed as:
Analysis detection is carried out according to above formula, and according to multiple samples, is divided by two-stage spanning forest algorithm, and
After detection, the recall rate of target reaches 100%, false alarm rate 48%.
As shown in figure 14, the embodiment of the present invention also proposes a kind of sonar image segmenting device, comprising:
Module 21 is established, for establishing non-directed graph using sonar initial data;
Divide module 22, for passing through minimum spanning tree and maximum spanning tree simultaneously to described undirected in the non-directed graph
Figure is split, wherein background area is marked off in the non-directed graph by the minimum spanning tree, by described maximum raw
Cheng Shu marks off target area in the non-directed graph.
The embodiment of the present invention uses sonar initial data in cutting procedure, does not lose the information of target maximumlly, subtracts
Influence of the small noise for cutting procedure;The it is proposed of the embodiment of the present invention in the non-directed graph simultaneously by minimum spanning tree and
Maximum spanning tree is split (the two poles of the earth spanning forest algorithm) to the non-directed graph, has taken into account the fluctuation of sonar datum target field gradient
Greatly, the unconspicuous feature of background area Fluctuation of gradient has obtained the accurate segmentation effect for target detection.
In one embodiment, described to establish module 21, it is used for:
Point all in the sonar initial data is constituted into point set V, by connection phases all in the sonar initial data
The collection E when constituting of adjacent two o'clock, the difference of the data value of adjacent two o'clock are the weight on side, point set V and side collection E constitute non-directed graph G (E,
V)。
In one embodiment, divide module 22, be used for:
The weight on side in the non-directed graph is sorted from small to large, successively the side in the non-directed graph is merged and is sentenced
It is disconnected, meet in two trees that two vertex on the side that currently grown are not belonging to same tree and described two vertex are adhered to separately
When minimum spanning tree similarity Condition, two trees of the side connection that currently grow are merged.
In one embodiment, divide module 22, be used for:
The weight on side in the non-directed graph is sorted from large to small;
In maximum spanning tree initial growth, determine that the side that currently grown belongs to target area;
Same tree and adhere to separately two of described two vertex are not belonging on two vertex on the side that currently grown
When tree meets maximum spanning tree similarity Condition, two trees of the side connection that currently grow are merged.
In one embodiment, divide module 22, be also used to: in the minimum spanning tree and maximum spanning tree growth course
In, to the information real-time update of each tree, the content of update includes at least one following:
The quantity for the point for including in tree;
The boundary position information of tree;
The center position information of tree;
The class label information of tree.
In one embodiment, described device further include:
Merging module 23, for merging overdivided region.
In one embodiment, the merging module 23, is used for:
The center for obtaining each region merges the linear areas detected according to the center.
In one embodiment, described device further include:
Detection module 24 is detected for carrying out size detection and/or conspicuousness to target area, will test by target
Region will test unsanctioned target area as false target as real goal.
As shown in figure 15, the embodiment of the present invention also provides a kind of sonar image splitting equipment, comprising: memory 31, processing
Device 32 and it is stored in the computer program that can be run on memory 31 and on processor 32, which is characterized in that the processor
The sonar image dividing method is realized when executing described program.
The embodiment of the present invention also proposes a kind of computer readable storage medium, is stored with computer executable instructions, described
Above-mentioned sonar image dividing method is realized when computer executable instructions are executed by processor.
It will appreciated by the skilled person that whole or certain steps, system, dress in method disclosed hereinabove
Functional module/unit in setting may be implemented as software, firmware, hardware and its combination appropriate.In hardware embodiment,
Division between the functional module/unit referred in the above description not necessarily corresponds to the division of physical assemblies;For example, one
Physical assemblies can have multiple functions or a function or step and can be executed by several physical assemblies cooperations.Certain groups
Part or all components may be implemented as by processor, such as the software that digital signal processor or microprocessor execute, or by
It is embodied as hardware, or is implemented as integrated circuit, such as specific integrated circuit.Such software can be distributed in computer-readable
On medium, computer-readable medium may include computer storage medium (or non-transitory medium) and communication media (or temporarily
Property medium).As known to a person of ordinary skill in the art, term computer storage medium is included in for storing information (such as
Computer readable instructions, data structure, program module or other data) any method or technique in the volatibility implemented and non-
Volatibility, removable and nonremovable medium.Computer storage medium include but is not limited to RAM, ROM, EEPROM, flash memory or its
His memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storages, magnetic holder, tape, disk storage or other
Magnetic memory apparatus or any other medium that can be used for storing desired information and can be accessed by a computer.This
Outside, known to a person of ordinary skill in the art to be, communication media generally comprises computer readable instructions, data structure, program mould
Other data in the modulated data signal of block or such as carrier wave or other transmission mechanisms etc, and may include any information
Delivery media.
Claims (12)
1. a kind of sonar image dividing method, comprising:
Non-directed graph is established using sonar initial data;
The non-directed graph is split by minimum spanning tree and maximum spanning tree simultaneously in the non-directed graph, wherein logical
It crosses the minimum spanning tree and marks off background area in the non-directed graph, through the maximum spanning tree in the non-directed graph
Mark off target area.
2. the method according to claim 1, wherein described establish non-directed graph using sonar initial data, comprising:
Point all in the sonar initial data is constituted into point set V, by connections adjacent two all in the sonar initial data
The collection E when constituting of point, the difference of the data value of adjacent two o'clock are the weight on side, and point set V and side collection E constitute non-directed graph G (E, V).
3. the method according to claim 1, wherein being divided by the minimum spanning tree the non-directed graph
It cuts, comprising:
The weight on side in the non-directed graph is sorted from small to large, judgement successively is merged to the side in the non-directed graph,
Two trees that two vertex on the side that currently grown are not belonging to same tree and described two vertex are adhered to separately meet minimum
When spanning tree similarity Condition, two trees of the side connection that currently grow are merged.
4. according to the method described in claim 3, it is characterized in that, two trees meet minimum spanning tree similarity Condition packet
It includes:
Dmin(C1,C2)<min(Dmin(C1)+d1,Dmin(C2)+d2)
Wherein,
Dmin(C1,C2) be the side that currently grown weight minimum value, min (Dmin(C1)+d1,Dmin(C2)+d2) table
Show and takes Dmin(C1)+d1And Dmin(C2)+d2In minimum value, Dmin(C1) it is tree C1The weight maximum value on middle side, Dmin(C2) it is tree C2
The weight maximum value on middle side, size (C1) it is tree C1In number of nodes, size (C2) it is tree C2In number of nodes, ε be it is described undirected
The mean value of all side right weights in figure.
5. the method according to claim 1, wherein being divided by the maximum spanning tree the non-directed graph
It cuts, comprising:
The weight on side in the non-directed graph is sorted from large to small;
In maximum spanning tree initial growth, determine that the side that currently grown belongs to target area;
In two trees symbol that two vertex on the side that currently grown are not belonging to same tree and described two vertex are adhered to separately
When closing maximum spanning tree similarity Condition, two trees of the side connection that currently grow are merged.
6. according to the method described in claim 5, determination is current it is characterized in that, described in maximum spanning tree initial growth
The side grown belongs to target area, comprising:
In the maximum spanning tree initial growth, it is pre- to determine that the maximum value on the side vertex Zhong Liangge that currently grown is greater than
If threshold value.
7. according to the method described in claim 5, it is characterized in that, two trees meet maximum spanning tree similarity Condition packet
It includes:
Dmax(C1,C2) > max (Dmax(C1)-k1,Dmax(C2)-k2)
Wherein,
Dmax(C1,C2) be the side that currently grown weight maximum value, max (Dmax(C1)-k1,Dmax(C2)-k2) table
Show and takes Dmax(C1)-k1And Dmax(C2)-k2In maximum value, Dmax(C1) it is tree C1The weight minimum value on middle side, Dmax(C2) it is tree C2
The weight minimum value on middle side, size (C1) it is tree C1In number of nodes, size (C2) it is tree C2In number of nodes,Indicate tree C1The weight limit on middle side and the difference of minimal weight,Indicate tree C2The weight limit on middle side and the difference of minimal weight.
8. the method according to claim 1, wherein the minimum spanning tree and maximum spanning tree are grown simultaneously,
The method also includes: in the minimum spanning tree and maximum spanning tree growth course, to the information real-time update of each tree,
The content of update includes at least one following:
The quantity for the point for including in tree;
The boundary position information of tree;
The center position information of tree;
The class label information of tree.
9. method described according to claim 1~any one of 8, which is characterized in that it is described in the non-directed graph simultaneously
After being split by minimum spanning tree and maximum spanning tree to the non-directed graph, further includes:
Merge overdivided region.
10. according to the method described in claim 9, it is characterized in that, the merging overdivided region, comprising:
The center for obtaining each target area merges the linear areas detected according to the center.
11. according to the method described in claim 9, it is characterized in that, the method is also wrapped after the merging overdivided region
Include: size detection and/or conspicuousness being carried out to target area and detected, will test by target area as real goal, general
Unsanctioned target area is detected as false target.
12. a kind of sonar image splitting equipment, comprising: memory, processor and storage are on a memory and can be on a processor
The computer program of operation, which is characterized in that the processor is realized when executing described program as any in claim 1~11
One sonar image dividing method.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811435393.3A CN109559321A (en) | 2018-11-28 | 2018-11-28 | A kind of sonar image dividing method and equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811435393.3A CN109559321A (en) | 2018-11-28 | 2018-11-28 | A kind of sonar image dividing method and equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN109559321A true CN109559321A (en) | 2019-04-02 |
Family
ID=65867725
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811435393.3A Pending CN109559321A (en) | 2018-11-28 | 2018-11-28 | A kind of sonar image dividing method and equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109559321A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110675410A (en) * | 2019-09-25 | 2020-01-10 | 江苏海洋大学 | Side-scan sonar sunken ship target unsupervised detection method based on selective search algorithm |
CN111784698A (en) * | 2020-07-02 | 2020-10-16 | 广州信瑞医疗技术有限公司 | Image self-adaptive segmentation method and device, electronic equipment and storage medium |
CN113379710A (en) * | 2021-06-18 | 2021-09-10 | 上海大学 | Underwater target sonar accurate measurement system and method |
CN114663434A (en) * | 2022-05-25 | 2022-06-24 | 国家***北海海洋技术保障中心 | Shadow discrimination method of side-scan sonar image |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101286229A (en) * | 2008-05-05 | 2008-10-15 | 哈尔滨工程大学 | Sonar image self-adapting division method based on stratified MRF |
CN105427342A (en) * | 2015-11-17 | 2016-03-23 | 中国电子科技集团公司第三研究所 | Method and system for detecting and tracking underwater small-target sonar image target |
CN105809672A (en) * | 2016-03-03 | 2016-07-27 | 浙江大学 | Super pixels and structure constraint based image's multiple targets synchronous segmentation method |
CN106127782A (en) * | 2016-06-30 | 2016-11-16 | 北京奇艺世纪科技有限公司 | A kind of image partition method and system |
CN108305258A (en) * | 2018-01-31 | 2018-07-20 | 成都快眼科技有限公司 | A kind of superpixel segmentation method, system and storage device based on minimum spanning tree |
-
2018
- 2018-11-28 CN CN201811435393.3A patent/CN109559321A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101286229A (en) * | 2008-05-05 | 2008-10-15 | 哈尔滨工程大学 | Sonar image self-adapting division method based on stratified MRF |
CN105427342A (en) * | 2015-11-17 | 2016-03-23 | 中国电子科技集团公司第三研究所 | Method and system for detecting and tracking underwater small-target sonar image target |
CN105809672A (en) * | 2016-03-03 | 2016-07-27 | 浙江大学 | Super pixels and structure constraint based image's multiple targets synchronous segmentation method |
CN106127782A (en) * | 2016-06-30 | 2016-11-16 | 北京奇艺世纪科技有限公司 | A kind of image partition method and system |
CN108305258A (en) * | 2018-01-31 | 2018-07-20 | 成都快眼科技有限公司 | A kind of superpixel segmentation method, system and storage device based on minimum spanning tree |
Non-Patent Citations (3)
Title |
---|
YUBING BAI 等: "A fast graph based method for object segmentation in sidescan sonar image", 《2018 7TH INTERNATIONAL SYMPOSIUM ON NEXT GENERATION ELECTRONICS (ISNE)》 * |
朱秀昌,刘峰,胡栋编著: "《数字图像处理与图像信息》", 31 August 2016, 北京邮电大学出版社 * |
王兴梅,时胜国著: "《基于水平集的水下声呐图像目标检测》", 30 June 2017, 哈尔滨工程大学出版社 * |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110675410A (en) * | 2019-09-25 | 2020-01-10 | 江苏海洋大学 | Side-scan sonar sunken ship target unsupervised detection method based on selective search algorithm |
CN110675410B (en) * | 2019-09-25 | 2023-06-13 | 江苏海洋大学 | Non-supervision detection method for side-scan sonar sunken ship target based on selective search algorithm |
CN111784698A (en) * | 2020-07-02 | 2020-10-16 | 广州信瑞医疗技术有限公司 | Image self-adaptive segmentation method and device, electronic equipment and storage medium |
CN113379710A (en) * | 2021-06-18 | 2021-09-10 | 上海大学 | Underwater target sonar accurate measurement system and method |
CN113379710B (en) * | 2021-06-18 | 2024-02-02 | 上海大学 | Underwater target sonar accurate measurement system and method |
CN114663434A (en) * | 2022-05-25 | 2022-06-24 | 国家***北海海洋技术保障中心 | Shadow discrimination method of side-scan sonar image |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102210715B1 (en) | Method, apparatus and device for determining lane lines in road | |
CN109559321A (en) | A kind of sonar image dividing method and equipment | |
CN111681197B (en) | Remote sensing image unsupervised change detection method based on Siamese network structure | |
CN106204555B (en) | A kind of optic disk localization method of combination Gbvs model and phase equalization | |
CN107346545A (en) | Improved confinement growing method for the segmentation of optic cup image | |
CN110517193A (en) | A kind of bottom mounted sonar Processing Method of Point-clouds | |
US20100265267A1 (en) | Analyzing pixel data by imprinting objects of a computer-implemented network structure into other objects | |
CN111985376A (en) | Remote sensing image ship contour extraction method based on deep learning | |
CN109685768A (en) | Lung neoplasm automatic testing method and system based on lung CT sequence | |
CN104217422B (en) | A kind of sonar image detection method of adaptive narrow-band level set | |
CN115097442B (en) | Water surface environment map construction method based on millimeter wave radar | |
Bartesaghi et al. | An energy-based three-dimensional segmentation approach for the quantitative interpretation of electron tomograms | |
Cheng et al. | Building boundary extraction from high resolution imagery and lidar data | |
CN109039503A (en) | A kind of frequency spectrum sensing method, device, equipment and computer readable storage medium | |
CN108491786A (en) | A kind of method for detecting human face based on hierarchical network and Cluster merging | |
CN110853011A (en) | Method for constructing convolutional neural network model for pulmonary nodule detection | |
CN112819958B (en) | Engineering geological mapping method and system based on three-dimensional laser scanning | |
CN112215842A (en) | Malignant nodule edge detection image processing method based on benign thyroid template | |
CN109003275A (en) | The dividing method of weld defect image | |
CN115393734A (en) | SAR image ship contour extraction method based on fast R-CNN and CV model combined method | |
CN117953314B (en) | Multi-dimensional feature optimization ocean substrate classification method and system | |
CN109493333A (en) | Ultrasonic Calcification in Thyroid Node point extraction algorithm based on convolutional neural networks | |
CN110307903A (en) | A kind of method of the contactless temperature dynamic measurement of poultry privileged site | |
CN108670301A (en) | A kind of backbone transverse process localization method based on ultrasonic image | |
CN112258536B (en) | Integrated positioning and segmentation method for calluses and cerebellum earthworm parts |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190402 |
|
RJ01 | Rejection of invention patent application after publication |