CN110321905A - Abnormal area detection method, system and associated component based on semantic segmentation - Google Patents
Abnormal area detection method, system and associated component based on semantic segmentation Download PDFInfo
- Publication number
- CN110321905A CN110321905A CN201910624645.5A CN201910624645A CN110321905A CN 110321905 A CN110321905 A CN 110321905A CN 201910624645 A CN201910624645 A CN 201910624645A CN 110321905 A CN110321905 A CN 110321905A
- Authority
- CN
- China
- Prior art keywords
- region
- image
- sample image
- saliency maps
- abnormal area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The abnormal area detection method based on semantic segmentation that the invention discloses a kind of, comprising: obtain sample image, and extract the Saliency maps of each sample image；Each training sample is sequentially input into the preset convolutional neural networks model for being used for semantic segmentation, convolutional neural networks model is trained；It wherein, include a sample image for being added to label and the Saliency maps of the sample image in each training sample；The Saliency maps of testing image and testing image are input in trained convolutional neural networks model, obtain the output of convolutional neural networks model as a result, and determining the abnormal area in testing image according to the label in output result.Using the scheme of the application, can be to avoid the abnormal area that can only detect limited kinds the case where, that is, false dismissal probability is reduced.Present invention also provides a kind of abnormal area detection system and associated component based on semantic segmentation have relevant art effect.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to a kind of abnormal area detection side based on semantic segmentation
Method, system and associated component.
Background technique
The semantic segmentation of image, expression is to be split using computer according to the semanteme of image.For example, input
There are house in picture, trees, road is exported house in result, trees by semantic segmentation, road different colours into
Rower note, that is, mark off respective profile.Semantic segmentation drives in GIS-Geographic Information System, unmanned vehicle, the neck such as medical imaging analysis
Domain is widely used, and one of the key problem as computer vision, and importance is also more and more prominent.
When carrying out the segmentation of profile using semantic segmentation technology, need that different marks is arranged for different things in advance
Label, such as different labels is arranged in respectively house, trees, road in training neural network, so that the nerve net after training
Network can identify and be partitioned into the house in testing image, trees and road.And if necessary to utilize semantic segmentation technology will be different
Normal region segmentation goes out, and needs to add label naturally for abnormal area and is trained, still, due in practical application, generation
The type of abnormal conditions is very more, just can not be by this kind of exception if a certain abnormal conditions are not provided with label and are trained
Abnormal area when happening is partitioned into, namely increases omission factor, in addition, label is excessively also unfavorable for a certain extent
It accurately identifies.Therefore, current semantic segmentation is usually the segmentation for being directed to the things with known fixed character and carrying out.
And when carrying out the abnormality detection of remote sensing images in traditional scheme, correlative study is less, mainly by extracting target
Spectrum is detected further according to the prior information in database, and standard, speed, practicability of detection etc. are all far away from use
The semantic segmentation technology of machine learning.
In conclusion avoiding to detect limited how when carrying out the detection of abnormal area using semantic segmentation technology
The case where abnormal area of type, reduces false dismissal probability, is current those skilled in the art technical problem urgently to be solved.
Summary of the invention
The object of the present invention is to provide a kind of abnormal area detection method, system and associated component based on semantic segmentation,
The case where to avoid the abnormal area that can only detect limited kinds, reduces false dismissal probability.
In order to solve the above technical problems, the invention provides the following technical scheme:
A kind of abnormal area detection method based on semantic segmentation, comprising:
Sample image is obtained, and extracts the Saliency maps of each Zhang Suoshu sample image；
Each training sample is sequentially input into the preset convolutional neural networks model for being used for semantic segmentation, to described
Convolutional neural networks model is trained；It wherein, include the sample image for being added to label in each training sample
And the Saliency maps of the sample image；
The Saliency maps of testing image and the testing image are input to the trained convolutional neural networks
In model, the output of the convolutional neural networks model is obtained as a result, and determining institute according to the label in the output result
State the abnormal area in testing image.
Preferably, the Saliency maps for extracting each Zhang Suoshu sample image, comprising:
The Saliency maps of each Zhang Suoshu sample image are extracted by RC algorithm.
Preferably, the Saliency maps that each Zhang Suoshu sample image is extracted by RC algorithm, comprising:
For each sample image, which is divided into multiple regions, and it is straight for each region to establish color
Fang Tu；
Pass throughThe saliency value for calculating each region obtains the conspicuousness of the sample image
Figure；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith
Region riColor distance,nkFor region rkIn color category
Number；niFor region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkKind color category
The probability of middle appearance；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D
(ck,p,ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement.
Preferably, the Saliency maps that each Zhang Suoshu sample image is extracted by RC algorithm, comprising:
For each sample image, which is divided into multiple regions, and it is straight for each region to establish color
Fang Tu；
Pass throughThe saliency value for calculating each region, obtains
The Saliency maps of the sample image out；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith
Region riColor distance,nkFor region rkIn color category
Number；niFor region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkKind color category
The probability of middle appearance；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D
(ck,p,ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement；Ds
(rk,ri) it is region rkWith region riSpace length, σsFor preset space weight affecting parameters.
Preferably, the acquisition sample image, and extract the Saliency maps of each Zhang Suoshu sample image, comprising:
Initial pictures are obtained, and each Zhang Suoshu initial pictures are cut into preset target sizes；
Image augmentation is carried out using one or more initial pictures after cutting；
The image collection that initial pictures after each image got by image augmentation and cutting are constituted, as
The sample image got, and extract the Saliency maps of each Zhang Suoshu sample image.
Preferably, the Saliency maps of testing image and the testing image are being input to the trained convolution
Before neural network model, further includes:
The size of the testing image is become to the positive integer times of the target sizes by full zero padding；
The testing image after full zero padding is cut, so that the size of every subgraph after cutting is equal to
The target sizes；
The Saliency maps by testing image and the testing image are input to the trained convolutional Neural
In network model, the output result of the convolutional neural networks model is obtained, comprising:
The Saliency maps of each subgraph and the subgraph are successively input to the trained convolutional Neural net
In network model, the output result that the convolutional neural networks model is directed to each subgraph is obtained；
The output result of each subgraph is placed on corresponding position, and the increased picture of full zero padding will be passed through
Plain position is cut, and the result after cutting is as the output result for the testing image got.
A kind of abnormal area detection system based on semantic segmentation, comprising:
Sample image Saliency maps extraction module for obtaining sample image, and extracts each Zhang Suoshu sample image
Saliency maps；
Model training module, for sequentially inputting each training sample into the preset convolutional Neural for being used for semantic segmentation
In network model, the convolutional neural networks model is trained；It wherein, include a Zhang Tianjia in each training sample
The sample image of label and the Saliency maps of the sample image；
Abnormal area detection module, for being input to the Saliency maps of testing image and the testing image by instruction
In the experienced convolutional neural networks model, the output of the convolutional neural networks model is obtained as a result, and according to the output
As a result the label in determines the abnormal area in the testing image.
Preferably, the sample image Saliency maps extraction module, is specifically used for:
Sample image is obtained, and extracts the Saliency maps of each Zhang Suoshu sample image by RC algorithm.
A kind of abnormal area detection device based on semantic segmentation, comprising:
Memory, for storing computer program；
Processor realizes the exception described in any of the above embodiments based on semantic segmentation for executing the computer program
The step of method for detecting area.
A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described
The abnormal area detection method described in any of the above embodiments based on semantic segmentation is realized when computer program is executed by processor
Step.
Using scheme provided by the embodiment of the present invention, the abnormal area detection based on semantic segmentation technology is realized.Tool
Body, the application has also extracted each in addition to inputting the sample image for being added to label into convolutional neural networks model
The Saliency maps of sample image, Saliency maps are able to reflect interregional color difference, and all kinds of abnormal areas usually has
One common feature, the color difference of the abnormal area and other regions that are as embodied by Saliency maps.Therefore, will add
The Saliency maps of tagged sample image and the sample image are inputted into convolutional neural networks model, convolutional neural networks
Model passes through after training, when, there are when abnormal area, the convolutional neural networks model based on semantic segmentation is just in picture to be measured
It can identify and be partitioned into the abnormal area.As can be seen that being used as due to being added to Saliency maps in the application for semantic point
Therefore the input of the convolutional neural networks model cut when carrying out the detection of abnormal area using semantic segmentation technology, can be kept away
Exempt from the case where can only detecting the abnormal area of limited kinds, that is, reduces false dismissal probability.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of implementation flow chart of the abnormal area detection method based on semantic segmentation in the present invention；
Fig. 2 is a kind of structural schematic diagram of the abnormal area detection system based on semantic segmentation in the present invention.
Specific embodiment
Core of the invention is to provide a kind of abnormal area detection method based on semantic segmentation, can be to avoid can only detect
The case where abnormal area of limited kinds, that is, reduce false dismissal probability.
In order to enable those skilled in the art to better understand the solution of the present invention, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.Obviously, described embodiments are only a part of the embodiments of the present invention, rather than
Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Under every other embodiment obtained, shall fall within the protection scope of the present invention.
Referring to FIG. 1, for a kind of implementation flow chart of the abnormal area detection method based on semantic segmentation in the present invention, it should
Abnormal area detection method based on semantic segmentation may comprise steps of:
Step S101: sample image is obtained, and extracts the Saliency maps of each sample image.
In general, can establish the neural network model for extracting Saliency maps, and by after training, utilizing the mind
Saliency maps are extracted through network model.
In a kind of specific embodiment of the invention, it is contemplated that if establishing the neural network for extracting Saliency maps
Model needs to carry out semantic segmentation in subsequent step by convolutional neural networks model, i.e. scheme needs two neural networks, meter
Calculation amount is very big, and calculating takes a long time, and when carrying out the detection of abnormal area, detection speed is low, does not determine in time
Abnormal area is unfavorable for carrying out abnormal conditions processing in time, therefore, extracts conspicuousness using RC algorithm in this kind of embodiment
Figure, i.e. step S101 can be with specifically: the Saliency maps that each sample image is extracted by RC algorithm are advantageously reduced and obtained
Take the time-consuming of Saliency maps.
There is also a variety of for RC algorithm, it is contemplated that reduces time-consuming purpose, in a kind of specific embodiment of the invention, leads to
The Saliency maps that RC algorithm extracts each sample image are crossed, following two step can be specifically included, are calculated more convenient.
First step: being directed to each sample image, which is divided into multiple regions, and be each region
Establish color histogram；
Second step: pass throughThe saliency value for calculating each region obtains the sample
The Saliency maps of image；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith
Region riColor distance,nkFor region rkIn color category
Number；niFor region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkKind color category
The probability of middle appearance；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D
(ck,p,ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement.
When sample image is divided into multiple regions, basic thought of dividing is using each pixel as the top of non-directed graph
Point, weight of the dissimilar degree as side between two pixels, connects the most authority on the side on two vertex in same area
The minimal weight on the important side for asking the vertex less than different zones carries out vertex conclusion and region merging technique in an iterative process.
For each region being partitioned into, the saliency value in the region, region r can be calculatedkIt can be any one
A region.ω(ri) indicate region riWeight, with region riIn pixel number emphasize the color contrast in big region, herein
The pixel number of description is region riIn weight ω (ri), big region described herein refers to that sample image is divided into multiple
In region, the biggish region of area, i.e., for region rkFor, saliency value S (rk) value influenced by other regions,
And ω (ri) biggish region, influence degree is also higher.
f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkThe probability occurred in kind color category；f(ci,q) be
Q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category, i.e. this kind of embodiment of the application
In, use the probability of occurrence of various colors in the color histogram in region as corresponding weight, to emphasize between primary color
Difference.
In a kind of specific embodiment of the invention, the conspicuousness of each sample image is extracted by RC algorithm
Figure, may include following two step:
Step 1: being directed to each sample image, which is divided into multiple regions, and establish for each region
Color histogram；
Step 2: pass throughCalculate the aobvious of each region
Work value obtains the Saliency maps of the sample image；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith
Region riColor distance,nkFor region rkIn color category
Number；niFor region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkKind color category
The probability of middle appearance；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D
(ck,p,ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement；Ds
(rk,ri) it is region rkWith region riSpace length, σsFor preset space weight affecting parameters.
In this kind of embodiment, it is contemplated that different zones have respective space characteristics, introduce space characteristics and carry out saliency value
Calculating, be conducive to the reasonability for improving calculated saliency value.Specifically, the region of neighbour should for any region
Increasing influences, and influence is then reduced in farther away region.
σsFor preset space weight affecting parameters, that is, control the intensity of space weight, σsIt is configured bigger, space right
The influence of value is just smaller, and will lead to can make biggish contribution to the saliency value of current region compared with the contrast of far region, usually
It can incite somebody to actionValue be set as 0.4, pixel coordinate then normalizes between [0,1].And it should be noted that the area Liang Ge
Space length between domain can be defined as the Euclidean distance of the center of gravity in two regions.
Step S102: each training sample is sequentially input into the preset convolutional neural networks model for being used for semantic segmentation
In, convolutional neural networks model is trained；It wherein, include the sample image for being added to label in each training sample
And the Saliency maps of the sample image.
Specifically, can be the convolutional neural networks model based on Seg-Net for semantic segmentation, convolutional neural networks
The specific composition of model can be set and be adjusted according to actual needs.For example, in a specific embodiment, being used for language
The convolutional neural networks model of justice segmentation is made of encoder and decoder, and importation includes RGB channel and a conspicuousness
Figure channel, the i.e. input of composition four-way.Encoder section can be by two groups (convolutional layers-convolutional layer-maximum pond layer) and three groups
(convolutional layer-convolutional layer-convolutional layer-maximum pond layer) is constituted, and decoder section can be by one group of (up-sampling layer-convolutional layer-volume
Lamination-convolutional layer), one group (up-sampling layer-convolutional layer-convolutional layer) and one group of (up-sampling layer-convolutional layer-convolutional layer-
Softmax multi-categorizer) it constitutes.Certainly, in other embodiments, it can according to need adjustment convolutional neural networks model,
Have no effect on implementation of the invention.
In a kind of specific embodiment of the invention, it is contemplated that in practical application collected testing image there may be
Situations such as various situation, such as image are fuzzy, and there are noises, in order to enable convolutional neural networks model also can be accurate
Ground carries out semantic segmentation to such testing image, determines abnormal area, and such picture can be added in training and is made
For training sample, i.e. step S101 can be specifically included:
Initial pictures are obtained, and each initial pictures are cut into preset target sizes；
Image augmentation is carried out using one or more initial pictures after cutting；
The image collection that initial pictures after each image got by image augmentation and cutting are constituted, as
The sample image got, and extract the Saliency maps of each sample image.
It is understood that being input in convolutional neural networks model when being trained to convolutional neural networks model
Each sample image need for identical size.And in practical application, the image shot, which would generally be much larger than, to be input to
Therefore size needed for convolutional neural networks model can cut the image shot.
Target sizes are size needed for being input to convolutional neural networks model.Such as in a kind of specific embodiment,
Image size needed for being input to convolutional neural networks model is 256*256, then can be after eliminating marginal point, initial
Generate x in image at random, y-coordinate, and centered on the coordinate, the image of lower 256*256 is cut, excluding marginal point can prevent
The case where image under cutting exceeds the range of initial pictures.
Image augmentation may include that rotation process is carried out to image, turning operation, Fuzzy Processing, illumination adjustment and addition
One or more combinations in the operation such as noise.
Step S103: the Saliency maps of testing image and testing image are input to trained convolutional neural networks
In model, the output of convolutional neural networks model is obtained as a result, and determining in testing image according to the label in output result
Abnormal area.
Different abnormal conditions have different characteristics, and still, the application is in view of having one when abnormal conditions occur
There are apparent colouring discriminations with peripheral region for general character, i.e. abnormal area, and the general character can be embodied by Saliency maps,
Therefore, convolutional neural networks model is trained using the Saliency maps of the sample image and sample image that are added to label
Later, convolutional neural networks model can carry out the identification of abnormal area based on the common feature.When existing in testing image
When abnormal area, output result will indicate the profile of abnormal area.
It differs in size it is understood that testing image is also likely to be present image in being input to convolutional neural networks model
It the case where image size needed, therefore, in a kind of specific embodiment of the invention, before step S103, can also wrap
It includes:
Step 1: the size of testing image is become to the positive integer times of target sizes by full zero padding；
Step 2: the testing image after full zero padding is cut, so that the size of every subgraph after cutting is equal
Equal to target sizes；
In this kind of embodiment, step S103 description: the Saliency maps of testing image and testing image are input to
In trained convolutional neural networks model, the output of convolutional neural networks model is obtained as a result, can be with specifically:
The Saliency maps of each subgraph and the subgraph are successively input to trained convolutional neural networks mould
In type, the output result that convolutional neural networks model is directed to each subgraph is obtained；
The output result of each subgraph is placed on corresponding position, and the increased picture of full zero padding will be passed through
Plain position is cut, and the result after cutting is as the output result for testing image got.
Target sizes are image size needed for being input to convolutional neural networks model, such as are arranged in previous embodiment
For the size of 256*256.Certainly, it should be noted that, then can be straight when the size of testing image is exactly equal to target sizes
Connect and be input in convolutional neural networks model, when testing image size be greater than target sizes, but be equal to target sizes it is just whole
When several times, it may not need full zero padding, directly execute the cutting operation of step 2.
Testing image after full zero padding is cut, every obtained subgraph has respective in former testing image
In belonging to position, therefore, obtain convolutional neural networks model for each subgraph output result after, by each Zhang Zitu
The output result of picture is placed on corresponding position, then cuts away before this by the increased location of pixels of full zero padding, just
The available output result to for testing image.
For example, the complete zero image graph A of a same size can be generated after carrying out full zero padding to testing image,
Successively the Saliency maps of each subgraph and the subgraph are input in trained convolutional neural networks model, are obtained
Convolutional neural networks model is directed to the output of each subgraph as a result, the output for any one subgraph is as a result, put
It sets in the corresponding position of figure A, figure B is cut into the size of former testing image, the knot of output by the figure B that detection that you can get it finishes
Fruit is the output result for being directed to testing image.
Using scheme provided by the embodiment of the present invention, the abnormal area detection based on semantic segmentation technology is realized.Tool
Body, applicant, although the type of abnormal conditions is varied, works as certain region and occurs it is considered that when being abnormal situation
When certain abnormal conditions, the color between abnormal area and normal region usually has more apparent difference, such as in forest
When on fire, region on fire and non-region on fire can have apparent colouring discrimination.Therefore, the application will be in addition to that will be added to label
Sample image is inputted into convolutional neural networks model, has also extracted the Saliency maps of each sample image, Saliency maps energy
Color difference between enough reflecting regionals.It is understood that being each different for the training sample that different abnormal conditions have occurred
Label is added in normal region, indicates that the region is abnormal area, although the type of these abnormal areas is different, but has one
A common feature, i.e., embodied by Saliency maps, the color difference in these regions and other regions, therefore, will be added
The sample image of label and the Saliency maps of the sample image are inputted into convolutional neural networks model, convolutional neural networks mould
Type passes through after training, when, there are when abnormal area, the convolutional neural networks model based on semantic segmentation just can in picture to be measured
It enough identifies and is partitioned into the abnormal area, it can the exceptions area in testing image is determined according to the label in output result
Domain.As can be seen that due to being added to Saliency maps in the application as the defeated of the convolutional neural networks model for semantic segmentation
Enter, it therefore, can be to avoid the exception that can only detect limited kinds when carrying out the detection of abnormal area using semantic segmentation technology
The case where region, reduces false dismissal probability.
Corresponding to above method embodiment, the embodiment of the invention also provides a kind of abnormal areas based on semantic segmentation
Detection system can correspond to each other reference with above.
It is shown in Figure 2, it is a kind of structural schematic diagram of the abnormal area detection system based on semantic segmentation in the present invention,
Include:
Sample image Saliency maps extraction module 201 for obtaining sample image, and extracts each sample image
Saliency maps；
Model training module 202, for sequentially inputting each training sample into the preset convolution for being used for semantic segmentation
In neural network model, convolutional neural networks model is trained；Wherein, include one in each training sample and be added to mark
The Saliency maps of the sample image of label and the sample image；
Abnormal area detection module 203, for being input to the Saliency maps of testing image and testing image by instruction
In experienced convolutional neural networks model, the output of convolutional neural networks model is obtained as a result, and according to the label in output result
Determine the abnormal area in testing image.
In a kind of specific embodiment of the invention, sample image Saliency maps extraction module 201 is specifically used for:
Sample image is obtained, and extracts the Saliency maps of each sample image by RC algorithm.
In a kind of specific embodiment of the invention, sample image Saliency maps extraction module 201 is specifically used for:
For each sample image, which is divided into multiple regions, and it is straight for each region to establish color
Fang Tu；
Pass throughThe saliency value for calculating each region obtains the significant of the sample image
Property figure；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith
Region riColor distance,nkFor region rkIn color category
Number；niFor region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkKind color category
The probability of middle appearance；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D
(ck,p,ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement.
In a kind of specific embodiment of the invention, sample image Saliency maps extraction module 201 is specifically used for:
For each sample image, which is divided into multiple regions, and it is straight for each region to establish color
Fang Tu；
Pass throughThe saliency value for calculating each region, obtains
The Saliency maps of the sample image out；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith
Region riColor distance,nkFor region rkIn color category
Number；niFor region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkKind color category
The probability of middle appearance；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D
(ck,p,ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement；Ds
(rk,ri) it is region rkWith region riSpace length, σsFor preset space weight affecting parameters.
In a kind of specific embodiment of the invention, sample image Saliency maps extraction module 201 is specifically used for:
Initial pictures are obtained, and each initial pictures are cut into preset target sizes；
Image augmentation is carried out using one or more initial pictures after cutting；
The image collection that initial pictures after each image got by image augmentation and cutting are constituted, as
The sample image got, and extract the Saliency maps of each sample image.
In a kind of specific embodiment of the invention, further includes testing image preprocessing module, is used for:
The size of testing image is become to the positive integer times of target sizes by full zero padding；
Testing image after full zero padding is cut, so that the size of every subgraph after cutting is equal to target
Size；
Abnormal area detection module 203, is specifically used for:
The Saliency maps of each subgraph and the subgraph are successively input to trained convolutional neural networks mould
In type, the output result that convolutional neural networks model is directed to each subgraph is obtained；
The output result of each subgraph is placed on corresponding position, and the increased picture of full zero padding will be passed through
Plain position is cut, and the result after cutting is as the output for testing image got as a result, and according to output result
In label determine the abnormal area in testing image.
Corresponding to above method and system embodiment, the embodiment of the invention also provides a kind of based on the different of semantic segmentation
Normal equipment for area detection equipment and a kind of computer readable storage medium can correspond to each other reference with above.
The abnormal area detection device based on semantic segmentation may include:
Memory, for storing computer program；
Processor realizes the exceptions area based on semantic segmentation in any of the above-described embodiment for executing computer program
The step of area detecting method.
It is stored with computer program on computer readable storage medium, is realized when computer program is executed by processor above-mentioned
The step of abnormal area detection method based on semantic segmentation in any embodiment.Computer-readable storage medium mentioned here
Matter includes random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable ROM, deposit
Any other form of storage medium well known in device, hard disk, moveable magnetic disc, CD-ROM or technical field.
It should also be noted that, herein, relational terms such as first and second and the like are used merely to one
Entity or operation are distinguished with another entity or operation, without necessarily requiring or implying between these entities or operation
There are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant are intended to contain
Lid non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including element.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
Used herein a specific example illustrates the principle and implementation of the invention, and above embodiments are said
It is bright to be merely used to help understand technical solution of the present invention and its core concept.It should be pointed out that for the common of the art
, without departing from the principle of the present invention, can be with several improvements and modifications are made to the present invention for technical staff, these
Improvement and modification are also fallen within the protection scope of the claims of the present invention.
Claims (10)
1. a kind of abnormal area detection method based on semantic segmentation characterized by comprising
Sample image is obtained, and extracts the Saliency maps of each Zhang Suoshu sample image；
Each training sample is sequentially input into the preset convolutional neural networks model for being used for semantic segmentation, to the convolution
Neural network model is trained；Wherein, include in each training sample the sample image for being added to label and
The Saliency maps of the sample image；
The Saliency maps of testing image and the testing image are input to the trained convolutional neural networks model
In, obtain the output of the convolutional neural networks model as a result, and according to the label in the output result determine it is described to
Abnormal area in altimetric image.
2. the abnormal area detection method according to claim 1 based on semantic segmentation, which is characterized in that described to extract
The Saliency maps of each Zhang Suoshu sample image, comprising:
The Saliency maps of each Zhang Suoshu sample image are extracted by RC algorithm.
3. the abnormal area detection method according to claim 2 based on semantic segmentation, which is characterized in that described to pass through RC
Algorithm extracts the Saliency maps of each Zhang Suoshu sample image, comprising:
For each sample image, which is divided into multiple regions, and establish color histogram for each region；
Pass throughThe saliency value for calculating each region obtains the Saliency maps of the sample image；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith region
riColor distance,nkFor region rkIn color category number；ni
For region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkOccur in kind color category
Probability；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D(ck,p,
ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement.
4. the abnormal area detection method according to claim 2 based on semantic segmentation, which is characterized in that described to pass through RC
Algorithm extracts the Saliency maps of each Zhang Suoshu sample image, comprising:
For each sample image, which is divided into multiple regions, and establish color histogram for each region；
Pass throughThe saliency value for calculating each region, obtains this
The Saliency maps of sample image；
Wherein, S (rk) indicate region rkSaliency value；ω(ri) indicate region riWeight；Dr(rk,ri) it is region rkWith region
riColor distance,nkFor region rkIn color category number；ni
For region riIn color category number；f(ck,p) it is pth kind color ck,pIn k-th of region rkIn nkOccur in kind color category
Probability；f(ci,q) it is q kind color ci,qIn ith zone riIn niThe probability occurred in kind color category；D(ck,p,
ci,q) it is region rkIn pth kind color ck,pWith region riIn q kind color ci,qBetween color distance measurement；Ds(rk,
ri) it is region rkWith region riSpace length, σsFor preset space weight affecting parameters.
5. the abnormal area detection method according to claim 1 based on semantic segmentation, which is characterized in that the acquisition sample
This image, and extract the Saliency maps of each Zhang Suoshu sample image, comprising:
Initial pictures are obtained, and each Zhang Suoshu initial pictures are cut into preset target sizes；
Image augmentation is carried out using one or more initial pictures after cutting；
The image collection that initial pictures after each image got by image augmentation and cutting are constituted, as acquisition
The sample image arrived, and extract the Saliency maps of each Zhang Suoshu sample image.
6. the abnormal area detection method according to claim 5 based on semantic segmentation, which is characterized in that will be to mapping
The Saliency maps of picture and the testing image are input to before the trained convolutional neural networks model, further includes:
The size of the testing image is become to the positive integer times of the target sizes by full zero padding；
The testing image after full zero padding is cut so that the size of every subgraph after cutting be equal to it is described
Target sizes；
The Saliency maps by testing image and the testing image are input to the trained convolutional neural networks
In model, the output result of the convolutional neural networks model is obtained, comprising:
The Saliency maps of each subgraph and the subgraph are successively input to the trained convolutional neural networks mould
In type, the output result that the convolutional neural networks model is directed to each subgraph is obtained；
The output result of each subgraph is placed on corresponding position, and the increased pixel position of full zero padding will be passed through
It sets and is cut, the result after cutting is as the output result for the testing image got.
7. a kind of abnormal area detection system based on semantic segmentation characterized by comprising
Sample image Saliency maps extraction module for obtaining sample image, and extracts the aobvious of each Zhang Suoshu sample image
Work property figure；
Model training module, for sequentially inputting each training sample into the preset convolutional neural networks for being used for semantic segmentation
In model, the convolutional neural networks model is trained；Wherein, include one in each training sample and be added to mark
The Saliency maps of the sample image of label and the sample image；
Abnormal area detection module, it is trained for being input to the Saliency maps of testing image and the testing image
In the convolutional neural networks model, the output of the convolutional neural networks model is obtained as a result, and according to the output result
In label determine the abnormal area in the testing image.
8. the abnormal area detection system according to claim 7 based on semantic segmentation, which is characterized in that the sample graph
As Saliency maps extraction module, it is specifically used for:
Sample image is obtained, and extracts the Saliency maps of each Zhang Suoshu sample image by RC algorithm.
9. a kind of abnormal area detection device based on semantic segmentation characterized by comprising
Memory, for storing computer program；
Processor, for executing the computer program to realize as claimed in any one of claims 1 to 6 based on semantic segmentation
Abnormal area detection method the step of.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program, when the computer program is executed by processor realize as it is as claimed in any one of claims 1 to 6 based on semantic segmentation
The step of abnormal area detection method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910624645.5A CN110321905A (en) | 2019-07-11 | 2019-07-11 | Abnormal area detection method, system and associated component based on semantic segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910624645.5A CN110321905A (en) | 2019-07-11 | 2019-07-11 | Abnormal area detection method, system and associated component based on semantic segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110321905A true CN110321905A (en) | 2019-10-11 |
Family
ID=68121963
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910624645.5A Pending CN110321905A (en) | 2019-07-11 | 2019-07-11 | Abnormal area detection method, system and associated component based on semantic segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110321905A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110766007A (en) * | 2019-10-28 | 2020-02-07 | 深圳前海微众银行股份有限公司 | Certificate shielding detection method, device and equipment and readable storage medium |
| CN111192679A (en) * | 2019-12-25 | 2020-05-22 | 上海联影智能医疗科技有限公司 | Method and device for processing image data exception and storage medium |
| CN111598098A (en) * | 2020-05-09 | 2020-08-28 | 河海大学 | Water gauge water line detection and effectiveness identification method based on full convolution neural network |
| CN111767831A (en) * | 2020-06-28 | 2020-10-13 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing image |
| CN112464745A (en) * | 2020-11-09 | 2021-03-09 | 中国科学院计算机网络信息中心 | Ground feature identification and classification method and device based on semantic segmentation |
| CN113112509A (en) * | 2021-04-12 | 2021-07-13 | 深圳思谋信息科技有限公司 | Image segmentation model training method and device, computer equipment and storage medium |
| CN116385807A (en) * | 2023-05-30 | 2023-07-04 | 南京信息工程大学 | Abnormal image sample generation method and device |
| CN116468205A (en) * | 2023-06-20 | 2023-07-21 | 青岛朗清众睿科技有限公司 | Method and system for monitoring environment-friendly detection quality of motor vehicle |
| CN117557570A (en) * | 2024-01-12 | 2024-02-13 | 中数智科(杭州)科技有限公司 | Rail vehicle abnormality detection method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122701A (en) * | 2017-03-03 | 2017-09-01 | 华南理工大学 | A kind of traffic route sign based on saliency and deep learning |
| US20190005380A1 (en) * | 2014-07-02 | 2019-01-03 | International Business Machines Corporation | Classifying features using a neurosynaptic system |
| CN109614973A (en) * | 2018-11-22 | 2019-04-12 | 华南农业大学 | Rice seedling and Weeds at seedling image, semantic dividing method, system, equipment and medium |
-
2019
- 2019-07-11 CN CN201910624645.5A patent/CN110321905A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190005380A1 (en) * | 2014-07-02 | 2019-01-03 | International Business Machines Corporation | Classifying features using a neurosynaptic system |
| CN107122701A (en) * | 2017-03-03 | 2017-09-01 | 华南理工大学 | A kind of traffic route sign based on saliency and deep learning |
| CN109614973A (en) * | 2018-11-22 | 2019-04-12 | 华南农业大学 | Rice seedling and Weeds at seedling image, semantic dividing method, system, equipment and medium |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110766007B (en) * | 2019-10-28 | 2023-09-22 | 深圳前海微众银行股份有限公司 | Certificate shielding detection method, device, equipment and readable storage medium |
| CN110766007A (en) * | 2019-10-28 | 2020-02-07 | 深圳前海微众银行股份有限公司 | Certificate shielding detection method, device and equipment and readable storage medium |
| CN111192679A (en) * | 2019-12-25 | 2020-05-22 | 上海联影智能医疗科技有限公司 | Method and device for processing image data exception and storage medium |
| CN111192679B (en) * | 2019-12-25 | 2024-04-19 | 上海联影智能医疗科技有限公司 | Method, device and storage medium for processing image data abnormality |
| CN111598098A (en) * | 2020-05-09 | 2020-08-28 | 河海大学 | Water gauge water line detection and effectiveness identification method based on full convolution neural network |
| CN111598098B (en) * | 2020-05-09 | 2022-07-29 | 河海大学 | Water gauge water line detection and effectiveness identification method based on full convolution neural network |
| CN111767831A (en) * | 2020-06-28 | 2020-10-13 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing image |
| CN111767831B (en) * | 2020-06-28 | 2024-01-12 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for processing image |
| CN112464745A (en) * | 2020-11-09 | 2021-03-09 | 中国科学院计算机网络信息中心 | Ground feature identification and classification method and device based on semantic segmentation |
| CN112464745B (en) * | 2020-11-09 | 2023-07-07 | 中国科学院计算机网络信息中心 | Feature identification and classification method and device based on semantic segmentation |
| CN113112509B (en) * | 2021-04-12 | 2023-07-04 | 深圳思谋信息科技有限公司 | Image segmentation model training method, device, computer equipment and storage medium |
| CN113112509A (en) * | 2021-04-12 | 2021-07-13 | 深圳思谋信息科技有限公司 | Image segmentation model training method and device, computer equipment and storage medium |
| CN116385807B (en) * | 2023-05-30 | 2023-09-12 | 南京信息工程大学 | Abnormal image sample generation method and device |
| CN116385807A (en) * | 2023-05-30 | 2023-07-04 | 南京信息工程大学 | Abnormal image sample generation method and device |
| CN116468205B (en) * | 2023-06-20 | 2023-09-08 | 青岛朗清众睿科技有限公司 | Method and system for monitoring environment-friendly detection quality of motor vehicle |
| CN116468205A (en) * | 2023-06-20 | 2023-07-21 | 青岛朗清众睿科技有限公司 | Method and system for monitoring environment-friendly detection quality of motor vehicle |
| CN117557570A (en) * | 2024-01-12 | 2024-02-13 | 中数智科(杭州)科技有限公司 | Rail vehicle abnormality detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321905A (en) | Abnormal area detection method, system and associated component based on semantic segmentation | |
| Huang et al. | Morphological building/shadow index for building extraction from high-resolution imagery over urban areas | |
| Heinzel et al. | Prior-knowledge-based single-tree extraction | |
| CN107622489A (en) | A kind of distorted image detection method and device | |
| CN108197587A (en) | A kind of method that multi-modal recognition of face is carried out by face depth prediction | |
| CN106980852B (en) | Based on Corner Detection and the medicine identifying system matched and its recognition methods | |
| Marinelli et al. | A novel approach to 3-D change detection in multitemporal LiDAR data acquired in forest areas | |
| CN106250845A (en) | Flame detecting method based on convolutional neural networks and device | |
| CN109977191B (en) | Problem map detection method, device, electronic equipment and medium | |
| CN111680632A (en) | Smoke and fire detection method and system based on deep learning convolutional neural network | |
| CN110413824A (en) | A kind of search method and device of similar pictures | |
| CN109165538A (en) | Bar code detection method and device based on deep neural network | |
| CN106897681A (en) | A kind of remote sensing images comparative analysis method and system | |
| CN107730515A (en) | Panoramic picture conspicuousness detection method with eye movement model is increased based on region | |
| CN108664838A (en) | Based on the monitoring scene pedestrian detection method end to end for improving RPN depth networks | |
| CN107909034A (en) | A kind of method for detecting human face, device and computer-readable recording medium | |
| CN108564092A (en) | Sunflower disease recognition method based on SIFT feature extraction algorithm | |
| CN109492665A (en) | Detection method, device and the electronic equipment of growth period duration of rice | |
| CN110188694B (en) | Method for identifying shoe wearing footprint sequence based on pressure characteristics | |
| CN114140665A (en) | Dense small target detection method based on improved YOLOv5 | |
| CN111291818B (en) | Non-uniform class sample equalization method for cloud mask | |
| CN112307984A (en) | Safety helmet detection method and device based on neural network | |
| CN108492288A (en) | The high score satellite image change detecting method of multiple dimensioned stratified sampling based on random forest | |
| CN106023199A (en) | Image analysis technology-based fume blackness intelligent detection method | |
| CN110532938A (en) | Papery operation page number recognition methods based on Faster-RCNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191011 |
|
| RJ01 | Rejection of invention patent application after publication |