CN110111346A - Remote sensing images semantic segmentation method based on parallax information - Google Patents
Remote sensing images semantic segmentation method based on parallax information Download PDFInfo
- Publication number
- CN110111346A CN110111346A CN201910399019.0A CN201910399019A CN110111346A CN 110111346 A CN110111346 A CN 110111346A CN 201910399019 A CN201910399019 A CN 201910399019A CN 110111346 A CN110111346 A CN 110111346A
- Authority
- CN
- China
- Prior art keywords
- parallax
- result
- remote sensing
- sensing images
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 82
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012360 testing method Methods 0.000 claims abstract description 83
- 238000001514 detection method Methods 0.000 claims abstract description 66
- 238000012549 training Methods 0.000 claims abstract description 41
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 40
- 238000012545 processing Methods 0.000 claims description 8
- 239000012141 concentrate Substances 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 4
- 238000007689 inspection Methods 0.000 claims description 4
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 2
- 235000013399 edible fruits Nutrition 0.000 claims description 2
- 230000003993 interaction Effects 0.000 abstract description 3
- 238000004064 recycling Methods 0.000 abstract 1
- 238000013135 deep learning Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 238000007637 random forest analysis Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 239000000155 melt Substances 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- VMXUWOKSQNHOCA-UKTHLTGXSA-N ranitidine Chemical compound [O-][N+](=O)\C=C(/NC)NCCSCC1=CC=C(CN(C)C)O1 VMXUWOKSQNHOCA-UKTHLTGXSA-N 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The present invention proposes a kind of remote sensing images semantic segmentation method based on parallax information, for solving the lower technical problem of segmentation precision existing in the prior art, realizes step are as follows: obtain training sample set and test sample collection；Training sample set is pre-processed；Parallax network is trained；Parallax detection is carried out to test sample collection；Obtain the parallax information of test sample collection；Semantic segmentation network is trained；Semantic detection is carried out to test sample collection；Preliminary semantic segmentation result is modified；Obtain final semantic segmentation result.The present invention carries out left and right consistency detection LRC to the parallax result of remote sensing images, and the parallax information for recycling detection to obtain is modified semantic segmentation result, and proposes a completely new semantic stripping and slicing fusion detection method, significantly improves the precision of semantic segmentation.It can be used for the practical applications such as ground quality detection, land use, urban planning, automatic Pilot, human-computer interaction, medical image identification.
Description
Technical field
The invention belongs to technical field of image processing, are related to a kind of remote sensing images semantic segmentation method, and in particular to a kind of
Remote sensing images semantic segmentation method based on parallax information can be applied to ground quality detection, land use, urban planning, drive automatically
Sail, human-computer interaction, medical image identification etc. fields.
Background technique
Semantic segmentation refers to according to color, gray scale and Texture eigenvalue, divides an image into several and mutually disjoints and have
There is the pixel region of certain certain semantic meaning, and identify the classification in each region, the pixel of the same area is endowed identical
Color, the final image for obtaining a width and there is pixel semantic tagger.
Remote sensing is to carry out the imaging of specific electromagnetic wave spectral coverage to the earth by the sensor on satellite, be with aeroplane photography
The a special kind of skill to grow up based on technology.It, can in a short time, to regional on a large scale on the earth by remote sensing
Multi-level, multi-angle of view observation is carried out, is the important means for obtaining environmental information and earth resource.Therefore, the letter of remote sensing image
Breath extractive technique is particularly important.It can be said that the final goal of remote sensing be just to be able to extract from image it is useful
Information gets knowledge.It, can be by image, semantic cutting techniques, by the specific objective in image in satellite remote sensing images
Division mark comes out, and the specific information in remote sensing images is extracted with this.Currently, as remote sensing images semantic segmentation technology is in geology
The extensive use in the fields such as detection, land use, urban planning, automatic Pilot, human-computer interaction, medical image identification, to semanteme
The requirement of segmentation precision is higher and higher.
Before deep learning is applied to computer vision field, researcher generally uses texture primitive forest
(TextonForest) or the conventional methods such as random forest (Random Forest) construct the classification for semantic segmentation
Device.Random forest is gone out the classification of sample by multiple decision tree independent predictions, and by voting all prediction classifications, poll is most
High classification is elected as final prediction result.Texture primitive forest then combines image texture characteristic building classifier, has
Improve to effect the precision of classification.But conventional method is relative complex, computationally intensive, the manual feature utilized needs field
Special knowledge takes time and effort, and cannot achieve real-time requirement, and too fining sometimes is so that can only be specific at some
It is used under scene, for possessing the image of complicated, changeable details, is extremely difficult to high-precision.
Currently, the semantic segmentation method based on deep learning is just receiving significant attention, main thought is to utilize to refer in advance
The true tag for determining training sample learns image by network, then does not occur with the model prediction training set of study
New samples, and provide the classification of new samples.Relatively traditional semantic segmentation method, the method based on deep learning has powerful
Learn suitable character representation ability for current problem, detection accuracy and speed, which have, to be increased substantially.For example, application is public
Cloth number be CN 107610141A, the patent application of entitled " a kind of remote sensing images semantic segmentation method based on deep learning ",
Disclose a kind of remote sensing images semantic segmentation method based on deep learning, this method be every kind of species Target Assignment rgb value and
Gray value obtains original remote sensing images, select species target and paint, gray processing and assign gray value processing, marked
Image is signed, data enhancing and edge extracting are carried out to original remote sensing images, the image after obtaining edge extracting；By original remote sensing
Image training sample after image and edge extracting is trained complete convolutional neural networks, obtains best semantic segmentation network
Test remote sensing images are inputted in best semantic segmentation network model, get semantic segmentation result images by model；For semanteme point
It cuts result images and carries out colouring processing, obtain final semantic segmentation result images.Although the complete convolution mind that this method proposes
It is realized instead of the penalty values computation layer in basic convolutional neural networks by convolutional calculation junior scholar through network using warp lamination
The remote sensing features image reconstruction that acquistion is arrived is to original image size, so that the precision of remote sensing images semantic segmentation is effectively increased,
There is stronger applicability for the segmentation problem of remote sensing images, still, there are still some shortcomingss for this method: due to the party
Method is extracted the marginal information of image as training data, to the accuracy requirement height of image border, therefore is suitable only for scene
Simple remote sensing images, for the remote sensing images of complex scene, marginal information is often fuzzy, non-closed, uses the party
Rule is easy the marginal information for making network excessively learn image, over-fitting occurs, causes semantic segmentation precision low.
Summary of the invention
It is an object of the invention to overcome above-mentioned the deficiencies in the prior art, a kind of remote sensing images based on parallax information are proposed
Semantic segmentation method, for solving the lower technical problem of segmentation precision existing in the prior art.
To achieve the above object, the technical solution that the present invention takes includes the following steps:
(1) training sample set and test sample collection are obtained:
Obtain the remote sensing that the satellite of two different locations is w × h to the s width different zones size that Same Scene is shot respectively
Image, s > 100, w > 1000, h > 1000, the shared c semantic classes of all pixels point of 2s width remote sensing images, c=1,2,
3 ..., the image of the same area of two satellites shooting is matched, obtains s remote sensing images pair, and select s therein1
A remote sensing images are to as training sample set, remaining s2A remote sensing images to as test sample collection,s2=s-
s1；
(2) training sample set is pre-processed:
The every width remote sensing images concentrated to training sample are sharpened processing, and to each image after Edge contrast into
The correction of row epipolar geom etry, obtains pretreated training sample set；
(3) parallax network is trained:
All remote sensing images that pretreated training sample is concentrated are to the view in input parallax network, after being trained
Poor network；
(4) parallax detection is carried out to test sample collection:
(4a) using belong in all remote sensing images pair that test sample collection includes a satellite shooting remote sensing images as
Test sample collection test1s, the remote sensing images of another satellite shooting are as test sample collection test2s；
(4b) is with test sample collection test1sOn the basis of, using the parallax network after training to test sample collection s2In it is every
A image obtains test to parallax detection is carried out1sParallax result disa,aiIt is i-th
Size be w × h parallax as a result, Indicate the parallax of j-th of pixel of i as a result, simultaneously
With test sample collection test2sOn the basis of, using the parallax network after training to test sample collection s2In each image to progress
Parallax detection, obtains test2sParallax result disb,biIt is w × h's for i-th of size
Parallax as a result, Indicate the parallax result of j-th of pixel of i；
(5) parallax information of test sample collection is obtained:
(5a) is to parallax result disaCarry out left and right consistency detection LRC:
(5a1) enables i=1, j=1；
(5a2) calculates parallax result disbIn with parallax result disa'sMatch the coordinate of pixel:
It is by coordinate value'sParallax valueCalculate parallax result disbIn withMatch pixel
Coordinate valueWith
(5a3) calculates parallax result disbIn with parallax result disa'sMatch the exhausted of pixel parallax value difference
To value:
Coordinates computed value isWithPoint parallax value absolute value of the differenceAnd coordinate
Value isWithPoint parallax value absolute value of the difference Indicates coordinate value is
Point parallax value,Indicates coordinate value isPoint parallax value；
(5a4) updates parallax result disaMiddle coordinate value isPoint parallax value
JudgementIt is whether true, δ ∈ (1,3), if so,It is not blocked, keepsIt is constant,
And step (5a5) is executed, otherwise, thenIt is blocked, in parallax result aiIn fromIt is horizontal to find first to the left not
It is blocked a littleFirst is found horizontally to the right not being blocked a littleIt enables
Indicates coordinate value isPoint parallax value,Indicates coordinate value isPoint parallax value, and execute step
(5a5)；
(5a5) obtains disaLeft and right consistency detection LRC result dis 'a:
Whether true judge j≤w × h, if so, enabling j=j+1, and executes step (5a2) and otherwise judge i≤s2Whether
It sets up, if so, enabling j=1, i=i+1, and executes step (5a2), otherwise, obtain parallax result disaLeft and right consistency detection
LRC result dis 'a；
(5b) is to parallax result disbCarry out left and right consistency detection LRC:
According to the method for step (5a) to parallax result disbLeft and right consistency detection LRC is carried out, parallax result dis is obtainedb
Left and right consistency detection LRC result disb', and by dis 'aAnd disb' parallax information the dis ' as test sample collection,puFor u-th of parallax information；
(6) semantic segmentation network is trained:
According to the sequence of Row Column or Column Row, the sliding window for being m × m by size, is step-length in training using n
It is slided, is obtained on every pretreated remote sensing images in sample setA figure
As block, and all image blocks are input in semantic segmentation network, the semantic segmentation network after being trained, m > 128, n >
64, m > n；
(7) Semantic detection is carried out to test sample collection:
(7a) be arranged using the long w of remote sensing images as x-axis, wide h be y-axis datum level, and by test sample concentrate every width
The semantic segmentation network that image rotates clockwise after the input training of each image after e degree relative to the datum level carries out stripping and slicing and melts
Detection is closed, dimension probability matrix matrix_e, e=0 identical with remote sensing images classification number c, 90,180 are obtained；
Probability matrix matrix_e is rotated e degree by (7b) counterclockwise, obtains the remote sensing images size phase for being w × h with size
Same, dimension is the probability matrix matrix_e ' of c, Indicate probability matrix
T-th of pixel of matrix_e ', e '=0,90,180, It indicatesBelong to
The probability value of semantic classes r；
(7c) is calculatedIn e '=0, the average value of 90,180 3 probability matrixs, obtain that size is w × h, dimension is the flat of c
Equal probability matrix matrix_avg,
Indicate t-th of pixelBelong to the probability value of semantic classes r；
(7d) calculates pixelSemantic classes r,It obtains just
Semantic segmentation result seg is walked,quIndicate u-th of preliminary semantic segmentation result；
(8) preliminary semantic segmentation result is modified:
(8a) is to parallax information puIt is clustered, obtains the parallax cluster result g of test sample collection,guIndicate u-th of parallax cluster result, guWith kuA parallax classification ku=1,2,
3,…；
(8b) divides parallax cluster result guIn pixel region area included by all parallax classificationsu, Indicate areauIn pixel included by z-th of parallax classification
Region；
(8c) passes through preliminary semantic segmentation result qu'sIn the most semantic classes pair of frequency of occurrenceIn
The semantic classes of all pixels point is replaced, and obtains the revised semantic segmentation result of test sample collection；
(9) final semantic segmentation result is obtained:
Each revised semantic segmentation of pair to test sample collection is every class semanteme mesh as a result, according to different classifications
Mark distribution rgb value, obtains the final semantic segmentation result of test sample collection.
Compared with prior art, the present invention having the advantage that
First, the present invention is due to carrying out parallax inspection to test sample collection first during obtaining semantic segmentation result
It surveys, and left and right consistency detection LRC, then the parallax information obtained by detection is carried out to test sample collection to parallax testing result
The primary segmentation result for carrying out Semantic detection acquisition is modified, finally to every class semanteme mesh of revised semantic segmentation result
Mark distribution rgb value, combine be all pixel scale parallax Detection task, sufficiently excavate remote sensing images multi-source information, avoid
The prior art detect and distribute semantic segmentation network to rgb value after image as final semantic segmentation result to segmentation essence
The influence of degree effectively improves the precision of semantic segmentation.
Second, the present invention carries out left and right consistency detection LRC to the parallax result that parallax network detects, and passes through left and right
The smaller value for finding first unshielding point parallax value is replaced the parallax value blocked a little, and blocking is a little only in a sub-picture
Middle appearance solves in the detection of remote sensing images parallax in the pixel that another piece image does not occur due to blocking a presence, nothing
Method finds match point, and then the problem for causing parallax precision low, obtains accurate parallax information, further improves semantic segmentation
Precision.
Third when the present invention carries out Semantic detection to remote sensing images, carries out stripping and slicing fusion inspection to remote sensing images to be detected
It surveys, first carries out remote sensing images to be detected up and down and bilateral symmetry, and sliding is carried out to the remote sensing images after symmetrical and takes block,
Then the central part that each image block is sent into the probability matrix that semantic network detects is intercepted, then splices each probability matrix
Central part, finally interception splicing posterior probability matrix central part, obtain the probability matrix of institute's detection image, both realized
Stripping and slicing small size detection to image, increases the detection accuracy to Small object, in turn avoids the erroneous detection to image border part,
Further improve the precision of semantic segmentation.
Detailed description of the invention
Fig. 1 is implementation flow chart of the invention；
Fig. 2 is the present invention to parallax result disaCarry out the implementation flow chart of left and right consistency detection LRC；
Fig. 3 is the implementation flow chart that the present invention carries out stripping and slicing fusion detection to remote sensing images.
Specific embodiment
In the following with reference to the drawings and specific embodiments, present invention is further described in detail.
Referring to Fig.1, the present invention includes the following steps:
Step 1) obtains training sample set and test sample collection:
Obtain the remote sensing that the satellite of two different locations is w × h to the s width different zones size that Same Scene is shot respectively
Image, s > 100, w > 1000, h > 1000, the shared c semantic classes of all pixels point of 2s width remote sensing images, c=1,2,
3 ..., the image of the same area of two satellites shooting is matched, obtains s remote sensing images pair, and select s therein1
A remote sensing images are to as training sample set, remaining s2A remote sensing images to as test sample collection,s2=s-
s1。
In the present embodiment, remote sensing images are obtained from the video that worldview-3 satellite is shot, s=4292, w
=1024, h=1024, c=5 randomly select 80% sample as training sample set s from s remote sensing images pair1, remaining
As test sample collection s2。
Step 2) pre-processes training sample set:
The every width remote sensing images concentrated to training sample are sharpened processing, and to each image after Edge contrast into
The correction of row epipolar geom etry, obtains pretreated training sample set.
The present embodiment is sharpened processing to every width remote sensing images, compensates the profile of image, enhances the edge and ash of image
The part for spending jump, is apparent from image, then to each image after Edge contrast to epipolar geom etry correction is carried out, so that often
Corresponding pixel points are in same horizontal line in the two images of a image pair, provide the training corrected for subsequent network training
Sample set.
Step 3) is trained parallax network:
All remote sensing images that pretreated training sample is concentrated are to the view in input parallax network, after being trained
Poor network.
Step 4) carries out parallax detection to test sample collection:
Step 4a) remote sensing images for belonging to a satellite shooting in all remote sensing images pair that test sample collection includes are made
For test sample collection test1s, the remote sensing images of another satellite shooting are as test sample collection test2s；
Step 4b) with test sample collection test1sOn the basis of, using the parallax network after training to test sample collection s2In
Each image obtains test to parallax detection is carried out1sParallax result disa,aiIt is i-th
A size be w × h parallax as a result, The parallax of j-th of pixel of i is indicated as a result, same
When with test sample collection test2sOn the basis of, using the parallax network after training to test sample collection s2In each image into
The detection of row parallax, obtains test2sParallax result disb,biIt is w × h for i-th of size
Parallax as a result, Indicate the parallax result of j-th of pixel of i.
The parallax information of step 5) acquisition test sample collection:
Step 5a) to parallax result disaLeft and right consistency detection LRC is carried out, implementation flow chart is as shown in Figure 2:
Step 5a1) enable i=1, j=1；
Step 5a2) calculate parallax result disbIn with parallax result disa'sMatch the coordinate of pixel:
It is by coordinate value'sParallax valueCalculate parallax result disbIn withMatch pixel
Coordinate valueWith
Step 5a3) calculate parallax result disbIn with parallax result disa'sMatch pixel parallax value difference
Absolute value:
Coordinates computed value isWithPoint parallax value absolute value of the differenceAnd it sits
Scale value isWithPoint parallax value absolute value of the difference Indicates coordinate value isPoint parallax value,Indicates coordinate value isPoint parallax value；
Step 5a4) update parallax result disaMiddle coordinate value isPoint parallax value
JudgementIt is whether true, δ ∈ (1,3), if so,It is not blocked, keepsIt is constant,
And step (5a5) is executed, otherwise, thenIt is blocked, in parallax result aiIn fromIt is horizontal to find first to the left not
It is blocked a littleFirst is found horizontally to the right not being blocked a littleIt enables Table
Show that coordinate value isPoint parallax value,Indicates coordinate value isPoint parallax value, and execute step
5a5)；
Step 5a5) obtain disaLeft and right consistency detection LRC result dis 'a:
Whether true judge j≤w × h, if so, enabling j=j+1, and execute step 5a2), otherwise, judge i≤s2Whether at
It is vertical, if so, enabling j=1, i=i+1, and execute step 5a2), otherwise, obtain parallax result disaLeft and right consistency detection LRC
As a result dis 'a；
Step 5b) to parallax result disbCarry out left and right consistency detection LRC:
With to parallax result disaThe step of carrying out left and right consistency detection LRC is the same, is only distinguished as a variable a and changes into
B, variable b change a into, according to the method for step 5a) to parallax result disbLeft and right consistency detection LRC is carried out, parallax knot is obtained
Fruit disbLeft and right consistency detection LRC result disb', and by dis 'aAnd disb' the parallax information as test sample collection
Dis ',puFor u-th of parallax information.
Left and right consistency detection LRC is carried out to the parallax result that parallax network detects, finds first by left and right
The smaller value of unshielding point parallax value is replaced the parallax value blocked a little, solve remote sensing images parallax detection in due to
A presence is blocked, the problem that can not be found match point, and then cause parallax precision low obtains accurate parallax information.
Step 6) is trained semantic segmentation network:
According to the sequence of Row Column or Column Row, the sliding window for being m × m by size, is step-length in training using n
It is slided, is obtained on every pretreated remote sensing images in sample setA figure
As block, and all image blocks are input in semantic segmentation network, the semantic segmentation network after being trained, m > 128, n >
64, m > n.
In the present embodiment, m=512, n=256, semantic segmentation are the detections of pixel scale, and each in remote sensing images
For the pixel size of classification target there is biggish difference, Small object is fuzzy, it is difficult to learn, by the image for acquiring small size
Block, amplification Small object allow network preferably to extract feature, improve the essence to Small object semantic segmentation in the ratio of image
Degree.
Step 7) carries out Semantic detection to test sample collection:
Step 7a) stripping and slicing fusion detection is carried out to remote sensing images, implementation flow chart is as shown in Figure 3:
Step 7a1) symmetrical above and below to remote sensing images to be detected progress:
Be arranged using the long w of remote sensing images as x-axis, wide h be y-axis datum level, and by test sample concentrate each image
Rotate clockwise e degree relative to the datum level, then be arranged e degree is rotated using every width after image long ω as x-axis, width ξ be y-axis, a left side
Inferior horn is the rectangular coordinate system of origin, by each image after rotation e degreeRegion is carried out with y
=ξ is the symmetry operation of axis,Region is carried out using y=0 as the symmetrical of axis
Operation, obtaining size isImage；
Step 7a2) bilateral symmetry is carried out to the remote sensing images after symmetrical above and below:
It is by sizeImageRegion is carried out with x=0
The symmetry operation of axis,Region is carried out using x=ω as the symmetrical behaviour of axis
Make, obtaining size isImage；
Step 7a3) it sliding is carried out to the remote sensing images after symmetrical takes block:
It is to sizeImage, according to Row Column or antecedent
Capable sequence afterwards, the sliding window for being λ × λ by size, withIt is slided for step-length, obtains image block set f, f=
{f1,f2,…,fo,…,fnum, foIndicate o-th of image block,
Step 7a4) Semantic detection is carried out to image block:
Semantic segmentation network after image block set f input training is detected, obtaining num size is the general of λ × λ
Rate matrix；
Step 7a5) intercept the probability matrix for detecting obtained image block:
Be arranged using num size for the probability matrix of λ × λ long λ as x-axis, width λ be y-axis, the right angle that the lower left corner is origin
Coordinate system intercepts each probability matrixRegion, obtain num interception after probability matrix collection
It closes；
Step 7a6) stitching image block probability matrix:
According to image block fwIt is in sizeThe opposite position of image
It sets, the probability matrix after num interception is spliced into the probability matrix that a size is (ω+λ) × (ξ+λ)；
Step 7a7) obtain the probability matrixs of detected remote sensing images:
Being arranged with the long ω+λ for the probability matrix that size is (ω+λ) × (ξ+λ) is x-axis, width ξ+λ is y-axis, the lower left corner is
The rectangular coordinate system of origin intercepts the region of (λ < x < λ+ω, the λ < y < λ+ξ) of probability matrix, and obtaining size is ω × ξ language
Justice divides dimension probability matrix matrix_e, e=0 identical with remote sensing images classification number c after network detection, 90,180.
In the present embodiment, λ=512, the stripping and slicing fusion detection utilized had both realized the stripping and slicing small size inspection to image
It surveys, increases the detection accuracy to Small object, in turn avoid further improving semantic segmentation to the erroneous detection of image border part
Precision.
Step 7b) probability matrix matrix_e rotated into e degree counterclockwise, it obtains with size being w × h remote sensing images size phase
Same, dimension is the probability matrix matrix_e ' of c, Indicate probability matrix
T-th of pixel of matrix_e ', e '=0,90,180, It indicatesBelong to
The probability value of semantic classes r；
Step 7c) it calculatesIn e '=0, the average value of 90,180 3 probability matrixs, obtaining size is w × h, dimension
For the average probability matrix matrix_avg of c,
Indicate t-th of pixelBelong to the probability value of semantic classes r；
Step 7d) calculate pixelSemantic classes r,?
To preliminary semantic segmentation result seg,quIndicate u-th of preliminary semantic segmentation result；
The pixel distributions of various forms of pictures is different, testing result also can difference, compared to by single network
It makes a policy, a possibility that mistake for the average value for taking multiple networks to make a policy is low, improves the accuracy of semantic segmentation.
Step 8) is modified preliminary semantic segmentation result:
Step 8a) to parallax information puIt is clustered, obtains the parallax cluster result g of test sample collection,guIndicate u-th of parallax cluster result, guWith kuA parallax classification ku=1,2,
3,…；
Step 8b) divide parallax cluster result guIn pixel region area included by all parallax classificationsu, Indicate areauIn pixel included by z-th of parallax classification
Region；
Step 8c) pass through preliminary semantic segmentation result qu'sIn the most semantic classes pair of frequency of occurrence
In the semantic classes of all pixels point be replaced, obtain the final semantic segmentation result of test sample collection.
Disparity computation result contains the atural objects such as building, trees parallax information abundant, the target category of same disparity value
It is bigger in the same category a possibility that, so parallax information has very great help for segmentation task.Therefore parallax result is used
Semantic results are modified.
Step 9) obtains final semantic segmentation result:
Each revised semantic segmentation of pair to test sample collection is every class semanteme mesh as a result, according to different classifications
Mark distribution rgb value, obtains the final semantic segmentation result of test sample collection.
Claims (2)
1. a kind of remote sensing images semantic segmentation method based on parallax information, which comprises the following steps:
(1) training sample set and test sample collection are obtained:
Obtain the remote sensing figure that the satellite of two different locations is w × h to the s width different zones size that Same Scene is shot respectively
Picture, s > 100, w > 1000, h > 1000, the shared c semantic classes of all pixels point of 2s width remote sensing images, c=1,2,
3 ..., the image of the same area of two satellites shooting is matched, obtains s remote sensing images pair, and select s therein1
A remote sensing images are to as training sample set, remaining s2A remote sensing images to as test sample collection,
(2) training sample set is pre-processed:
Processing is sharpened to every width remote sensing images that training sample is concentrated, and to each image after Edge contrast to progress pole
Line geometry correction, obtains pretreated training sample set；
(3) parallax network is trained:
All remote sensing images that pretreated training sample is concentrated are to the parallax net in input parallax network, after being trained
Network；
(4) parallax detection is carried out to test sample collection:
(4a) will belong to the remote sensing images of satellite shooting as test in all remote sensing images pair that test sample collection includes
Sample set test1s, the remote sensing images of another satellite shooting are as test sample collection test2s；
(4b) is with test sample collection test1sOn the basis of, using the parallax network after training to test sample collection s2In each figure
As obtaining test to parallax detection is carried out1sParallax result disa,aiFor i-th of size
For w × h parallax as a result, The parallax of j-th of pixel of i is indicated as a result, simultaneously to survey
Try sample set test2sOn the basis of, using the parallax network after training to test sample collection s2In each image to carry out parallax
Detection, obtains test2sParallax result disb,biThe parallax for being w × h for i-th of size
As a result, Indicate the parallax result of j-th of pixel of i；
(5) parallax information of test sample collection is obtained:
(5a) is to parallax result disaCarry out left and right consistency detection LRC:
(5a1) enables i=1, j=1；
(5a2) calculates parallax result disbIn with parallax result disa'sMatch the coordinate of pixel:
It is by coordinate value'sParallax valueCalculate parallax result disbIn withMatch the coordinate of pixel
ValueWith
(5a3) calculates parallax result disbIn with parallax result disa'sMatch pixel parallax value absolute value of the difference:
Coordinates computed value isWithPoint parallax value absolute value of the differenceAnd coordinate value isWithPoint parallax value absolute value of the difference Indicates coordinate value isPoint
Parallax value,Indicates coordinate value isPoint parallax value；
(5a4) updates parallax result disaMiddle coordinate value isPoint parallax value
JudgementIt is whether true, δ ∈ (1,3), if so,It is not blocked, keepsIt is constant, and hold
Row step (5a5), otherwise, thenIt is blocked, in parallax result aiIn fromHorizontal searching first to the left is not hidden
Catch pointFirst is found horizontally to the right not being blocked a littleIt enables It indicates to sit
Scale value isPoint parallax value,Indicates coordinate value isPoint parallax value, and execute step
(5a5)；
(5a5) obtains disaLeft and right consistency detection LRC result dis 'a:
Whether true judge j≤w × h, if so, enabling j=j+1, and executes step (5a2) and otherwise judge i≤s2It is whether true,
If so, enabling j=1, i=i+1, and step (5a2) is executed, otherwise, obtains parallax result disaLeft and right consistency detection LRC knot
Fruit dis 'a；
(5b) is to parallax result disbCarry out left and right consistency detection LRC:
According to the method for step (5a) to parallax result disbLeft and right consistency detection LRC is carried out, parallax result dis is obtainedbA left side
Right uniformity detects LRC result dis 'b, and by dis 'aWith dis 'bAs the parallax information dis ' of test sample collection,puFor u-th of parallax information；
(6) semantic segmentation network is trained:
According to the sequence of Row Column or Column Row, the sliding window for being m × m by size, is step-length in training sample using n
It concentrates and is slided on every pretreated remote sensing images, obtainedA image
Block, and all image blocks are input in semantic segmentation network, the semantic segmentation network after being trained, m > 128, n > 64, m
> n；
(7) Semantic detection is carried out to test sample collection:
(7a) be arranged using the long w of remote sensing images as x-axis, wide h be y-axis datum level, and by test sample concentrate each image
The semantic segmentation network after each image input training after rotating clockwise e degree relative to the datum level carries out stripping and slicing fusion inspection
It surveys, obtains dimension probability matrix matrix_e, e=0 identical with remote sensing images classification number c, 90,180；
Probability matrix matrix_e is rotated e degree by (7b) counterclockwise, obtains, dimension identical for the remote sensing images size of w × h as size
Degree is the probability matrix matrix_e ' of c, Indicate probability matrix
T-th of pixel of matrix_e ', e '=0,90,180, It indicatesBelong to
The probability value of semantic classes r；
(7c) is calculatedIn e '=0, the average value of 90,180 3 probability matrixs, obtain that size is w × h, dimension is the average general of c
Rate matrix matrix_avg, Indicate t-th of pixelBelong to the probability value of semantic classes r；
(7d) calculates pixelSemantic classes r,Obtain preliminary semanteme
Segmentation result seg,quIndicate u-th of preliminary semantic segmentation result；
(8) preliminary semantic segmentation result is modified:
(8a) is to parallax information puIt is clustered, obtains the parallax cluster result g of test sample collection,guIndicate u-th of parallax cluster result, guWith kuA parallax classification ku=1,2,
3,…；
(8b) divides parallax cluster result guIn pixel region area included by all parallax classificationsu, Indicate areauIn pixel included by z-th of parallax classification
Region；
(8c) passes through preliminary semantic segmentation result qu'sIn the most semantic classes pair of frequency of occurrenceIn it is all
The semantic classes of pixel is replaced, and obtains the revised semantic segmentation result of test sample collection；
(9) final semantic segmentation result is obtained:
Each revised semantic segmentation of pair to test sample collection is as a result, according to different classifications, for every class semantic objects point
With rgb value, the final semantic segmentation result of test sample collection is obtained.
2. the remote sensing images semantic segmentation method according to claim 1 based on parallax information, which is characterized in that step
Each image after each image that test sample is concentrated to be rotated clockwise relative to the datum level to e degree described in (7a) is defeated
Semantic segmentation network after entering training carries out stripping and slicing fusion detection, realizes step are as follows:
(7a1) carries out remote sensing images to be detected symmetrical above and below:
Be arranged the long ω of image after rotating e degree using every width of test sample collection as x-axis, width ξ be y-axis, the lower left corner is the straight of origin
Angular coordinate system, by each image after rotation e degreeRegion is carried out using y=ξ as the symmetrical behaviour of axis
Make,Region is carried out using y=0 as the symmetry operation of axis, and obtaining size isImage；
(7a2) carries out bilateral symmetry to the remote sensing images after symmetrical above and below:
It is by sizeImageRegion is carried out using x=0 as axis
Symmetry operation,Region obtain using x=ω as the symmetry operation of axis
It is to sizeImage；
(7a3) carries out sliding to the remote sensing images after symmetrical and takes block:
It is to sizeImage, according to Row Column or Column Row
Sequence, by size be λ × λ sliding window, withIt is slided for step-length, obtains image block set f, f={ f1,
f2,…,fo,…,fnum, foIndicate o-th of image block,
(7a4) carries out Semantic detection to image block:
Semantic segmentation network after image block set f input training is detected, the probability square that num size is λ × λ is obtained
Battle array；
The probability matrix for the image block that (7a5) interception detection obtains:
Be arranged using num size for the probability matrix of λ × λ long λ as x-axis, width λ be y-axis, the rectangular co-ordinate that the lower left corner is origin
System, intercepts each probability matrixRegion, obtain num interception after probability matrix set；
The probability matrix of (7a6) stitching image block:
According to image block foIt is in sizeThe relative position of image, will
Probability matrix after num interception is spliced into the probability matrix that a size is (ω+λ) × (ξ+λ)；
(7a7) obtains the probability matrix of detected remote sensing images:
Be arranged using size be (ω+λ) × (ξ+λ) probability matrix long ω+λ be x-axis, width ξ+λ is y-axis, the lower left corner as origin
Rectangular coordinate system, intercept the region of (λ < x < λ+ω, the λ < y < λ+ξ) of probability matrix, obtain size as ω × ξ semanteme point
Probability matrix matrix_e after cutting network detection.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910399019.0A CN110111346B (en) | 2019-05-14 | 2019-05-14 | Remote sensing image semantic segmentation method based on parallax information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910399019.0A CN110111346B (en) | 2019-05-14 | 2019-05-14 | Remote sensing image semantic segmentation method based on parallax information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110111346A true CN110111346A (en) | 2019-08-09 |
| CN110111346B CN110111346B (en) | 2020-12-08 |
Family
ID=67489883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910399019.0A Active CN110111346B (en) | 2019-05-14 | 2019-05-14 | Remote sensing image semantic segmentation method based on parallax information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110111346B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111292340A (en) * | 2020-01-23 | 2020-06-16 | 北京市商汤科技开发有限公司 | Semantic segmentation method, device, equipment and computer readable storage medium |
| CN112288650A (en) * | 2020-10-28 | 2021-01-29 | 武汉大学 | Multi-source remote sensing satellite image geometric and semantic integrated processing method and system |
| CN115294489A (en) * | 2022-06-22 | 2022-11-04 | 太原理工大学 | Semantic segmentation method and system for disaster video data |
| CN116129525A (en) * | 2023-01-24 | 2023-05-16 | 中国人民解放军陆军防化学院 | Respiratory protection training evaluation system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0997334A (en) * | 1995-07-24 | 1997-04-08 | Nec Corp | Method and device for three-dimensional reference image segmentation and object judging device |
| CN103237228A (en) * | 2013-04-28 | 2013-08-07 | 清华大学 | Time-space consistency segmentation method for binocular stereoscopic video |
| CN104469386A (en) * | 2014-12-15 | 2015-03-25 | 西安电子科技大学 | Stereoscopic video perception and coding method for just-noticeable error model based on DOF |
| CN108038866A (en) * | 2017-12-22 | 2018-05-15 | 湖南源信光电科技股份有限公司 | A kind of moving target detecting method based on Vibe and disparity map Background difference |
-
2019
- 2019-05-14 CN CN201910399019.0A patent/CN110111346B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0997334A (en) * | 1995-07-24 | 1997-04-08 | Nec Corp | Method and device for three-dimensional reference image segmentation and object judging device |
| CN103237228A (en) * | 2013-04-28 | 2013-08-07 | 清华大学 | Time-space consistency segmentation method for binocular stereoscopic video |
| CN104469386A (en) * | 2014-12-15 | 2015-03-25 | 西安电子科技大学 | Stereoscopic video perception and coding method for just-noticeable error model based on DOF |
| CN108038866A (en) * | 2017-12-22 | 2018-05-15 | 湖南源信光电科技股份有限公司 | A kind of moving target detecting method based on Vibe and disparity map Background difference |
Non-Patent Citations (2)
| Title |
|---|
| YAN LIU,ET AL.: "Efficient Patch-Wise Semantic Segmentation for Large-Scale Remote Sensing Images", 《SENSORS》 * |
| 吴止锾 等: "类别非均衡遥感图像语义分割的全卷积网络方法", 《光学学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111292340A (en) * | 2020-01-23 | 2020-06-16 | 北京市商汤科技开发有限公司 | Semantic segmentation method, device, equipment and computer readable storage medium |
| CN112288650A (en) * | 2020-10-28 | 2021-01-29 | 武汉大学 | Multi-source remote sensing satellite image geometric and semantic integrated processing method and system |
| CN112288650B (en) * | 2020-10-28 | 2021-07-20 | 武汉大学 | Multi-source remote sensing satellite image geometric and semantic integrated processing method and system |
| CN115294489A (en) * | 2022-06-22 | 2022-11-04 | 太原理工大学 | Semantic segmentation method and system for disaster video data |
| CN115294489B (en) * | 2022-06-22 | 2023-06-09 | 太原理工大学 | Semantic segmentation method and system for disaster video data |
| CN116129525A (en) * | 2023-01-24 | 2023-05-16 | 中国人民解放军陆军防化学院 | Respiratory protection training evaluation system and method |
| CN116129525B (en) * | 2023-01-24 | 2023-11-14 | 中国人民解放军陆军防化学院 | Respiratory protection training evaluation system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110111346B (en) | 2020-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110111346A (en) | Remote sensing images semantic segmentation method based on parallax information | |
| Jia et al. | Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot | |
| CN108596101B (en) | Remote sensing image multi-target detection method based on convolutional neural network | |
| CN106023157B (en) | A kind of micro- deformation data extracting method of mountain area earth's surface based on SAR image | |
| CN109961006A (en) | A kind of low pixel multiple target Face datection and crucial independent positioning method and alignment schemes | |
| CN101326514B (en) | Background removal in a live video | |
| CN109584213B (en) | Multi-target number selection tracking method | |
| Bi et al. | Rethinking camouflaged object detection: Models and datasets | |
| Kirthika et al. | Automated road network extraction using artificial neural network | |
| CN110378997A (en) | A kind of dynamic scene based on ORB-SLAM2 builds figure and localization method | |
| CN110827312B (en) | Learning method based on cooperative visual attention neural network | |
| CN106897681A (en) | A kind of remote sensing images comparative analysis method and system | |
| CN109635634A (en) | A kind of pedestrian based on stochastic linear interpolation identifies data enhancement methods again | |
| CN108846416A (en) | The extraction process method and system of specific image | |
| Xu et al. | Recognition of weeds in wheat fields based on the fusion of RGB images and depth images | |
| CN104182968A (en) | Method for segmenting fuzzy moving targets by wide-baseline multi-array optical detection system | |
| CN106570885A (en) | Background modeling method based on brightness and texture fusion threshold value | |
| CN110472081A (en) | A kind of cross-domain search method of shoes picture based on metric learning | |
| CN108256567A (en) | A kind of target identification method and system based on deep learning | |
| CN111191531A (en) | Rapid pedestrian detection method and system | |
| CN104933691B (en) | Image interfusion method based on the detection of phase spectrum vision significance | |
| CN109785359A (en) | A kind of video object detection method based on depth characteristic pyramid and tracking loss | |
| CN107423771B (en) | Two-time-phase remote sensing image change detection method | |
| Tarchoun et al. | Adversarial attacks in a multi-view setting: an empirical study of the adversarial patches inter-view transferability | |
| CN107292268A (en) | The SAR image semantic segmentation method of quick ridge ripple deconvolution Structure learning model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |