CN110008947A - A kind of silo Grain Quantity monitoring method and device based on convolutional neural networks - Google Patents
A kind of silo Grain Quantity monitoring method and device based on convolutional neural networks Download PDFInfo
- Publication number
- CN110008947A CN110008947A CN201910295466.1A CN201910295466A CN110008947A CN 110008947 A CN110008947 A CN 110008947A CN 201910295466 A CN201910295466 A CN 201910295466A CN 110008947 A CN110008947 A CN 110008947A
- Authority
- CN
- China
- Prior art keywords
- grain
- grain face
- image
- convolutional neural
- neural networks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 92
- 238000012544 monitoring process Methods 0.000 title claims abstract description 55
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 49
- 238000003709 image segmentation Methods 0.000 claims abstract description 32
- 230000008569 process Effects 0.000 claims abstract description 25
- 235000013339 cereals Nutrition 0.000 claims description 235
- 241000209094 Oryza Species 0.000 claims description 71
- 235000007164 Oryza sativa Nutrition 0.000 claims description 71
- 235000009566 rice Nutrition 0.000 claims description 71
- 238000012549 training Methods 0.000 claims description 53
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000012806 monitoring device Methods 0.000 claims description 6
- 238000007670 refining Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 11
- 238000012545 processing Methods 0.000 abstract description 10
- 230000011218 segmentation Effects 0.000 description 18
- 230000008859 change Effects 0.000 description 8
- 238000012360 testing method Methods 0.000 description 7
- 238000007689 inspection Methods 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000013316 zoning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/143—Sensing or illuminating at different wavelengths
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of silo Grain Quantity monitoring method and device based on convolutional neural networks, first acquire grain face image, then grain face image is input in the storehouse based on convolutional neural networks put up in Image Segmentation Model, identify grain face and reference line, the grain face in grain face image and the area between reference line are finally calculated, and calculates the error between preset area, obtains error amount, if error amount is greater than the error threshold of setting, determine that silo Grain Quantity changes.This method is the monitoring method for monitoring Grain Quantity variation automatically using image processing techniques.The monitoring accuracy and reliability of this method are higher, can effectively push grain depot intelligentized updating that process is transformed.In addition, can be combined with the detection method scanned based on infrared laser using this method, when finding grain face variation, that is, it infrared laser scanner can be used to carry out whole storehouse scanning, accurate grain volume obtained, to delay the service life of infrared laser.
Description
Technical field
The present invention relates to a kind of silo Grain Quantity monitoring method and device based on convolutional neural networks.
Background technique
Grain Stock Check generallys use direct measuring method at present, not only time and effort consuming, and be easy by it is subjective because
Element interference.In recent years, related scientific research personnel propose some advanced mensurations, the precision and intelligence detected with lift grain quantity
Energyization, the grain inventory inspection method such as based on pressure sensor, the grain inventory inspection method based on infrared laser scanning, base
In the grain inventory inspection method of radar detection and grain inventory inspection method based on ultrasound etc..Relative to traditional artificial
Grain inventory's inspection method, the above method not only improve the efficiency and detection accuracy taken an inventory of warehouses, also reduce and take an inventory of warehouses
Expense.But other than above-mentioned advantage, there is also some shortcomings for these methods, such as: the method pair based on pressure sensor
The laying of sensor is more demanding, and the sensitivity of sensor also can gradually degenerate with the time；And based on infrared laser scanning
Method cannot use, since the infrared laser service life limits in addition, using general laser scanner accurate scan for a long time
One time silo time overhead is bigger；And tower silo is confined to based on radar detection and method based on ultrasound, it is not suitable for
Other Grain Quantities such as the storehouse type of large storehouse detect, and detection reliability is poor.In addition, the above method is deployed in grain depot
Higher cost and not convenient for safeguarding.
Summary of the invention
The silo Grain Quantity monitoring method based on convolutional neural networks that the object of the present invention is to provide a kind of, to solve
The poor problem of the reliability of existing silo Grain Quantity monitoring method.The present invention also provides one kind to be based on convolutional neural networks
Silo Grain Quantity monitoring device, the reliability to solve the problems, such as existing silo Grain Quantity monitoring mode is poor.
To achieve the above object, the solution of the present invention includes:
A kind of silo Grain Quantity monitoring method based on convolutional neural networks, comprising the following steps:
(1) grain face image is acquired；The grain face image includes the reference line above grain face and grain face；
(2) the grain face image is input in the storehouse based on convolutional neural networks put up in Image Segmentation Model,
Identify grain face and reference line；
(3) grain face in the grain face image and the area between reference line are calculated, and is calculated between preset area
Error, obtain error amount, if the error amount be greater than setting error threshold, determine that silo Grain Quantity changes.
This programme carries out the automatic prison of silo Grain Quantity using Image Segmentation Model in the storehouse based on convolutional neural networks
It surveys, collected grain face image is handled by the way of image procossing, whether final process obtains silo Grain Quantity
It changes, this method is one and combines the image processing techniques based on convolutional neural networks to be supervised automatically using grain face image
The monitoring method for surveying Grain Quantity variation, realizes Grain Quantity safety monitoring function based on computer vision.Relative to existing
Monitoring method, this method do not need to put into too many professional equipment, it is only necessary to put into Image Acquisition and processing equipment, so
Carrying out processing to image afterwards can be realized as monitoring automatically, at low cost, and deployment is simple.Also, this method need to only detect grain face
Image has detection speed fast, low energy consumption by that can obtain whether silo Grain Quantity changes after processing
The advantages of.Therefore, the monitoring accuracy of this method and reliability are higher, can effectively push grain depot intelligentized updating that process is transformed.
In addition, can be combined with the detection method scanned based on infrared laser using this method, when finding grain face variation, that is, can be used
Infrared laser scanner carries out whole storehouse scanning, accurate grain volume is obtained, to delay the service life of infrared laser.
Further, in order to improve silo Grain Quantity monitoring reliability, Image Segmentation Model was built in the storehouse
Journey include: acquisition grain face sample image, in grain face sample image grain face and reference line be marked, generate training set；
Convolutional neural networks model is built, training set is input to convolutional neural networks model and is trained, image segmentation in storehouse is obtained
Model.
Further, in order to improve the reliability of Image Segmentation Model in storehouse, Image Segmentation Model is built in the storehouse
In the process, it to the grain face in grain face sample image and after reference line is marked, is generated by the way of image set enhancing
More massive training set.
Further, in order to improve the identification accuracy of grain face and reference line, in step (2), grain face image is identified
In grain face and the process of reference line include: size by grain face image scaling to image corresponding with training set, use
To storehouse in Image Segmentation Model grain face image is split, obtain initial grain face and reference line；Then, to getting
Initial grain face and reference line expanded respectively；Finally, refining respectively to grain face and reference line, identification is obtained
Grain face and reference line.
Further, in order to improve the identification accuracy of grain face and reference line, to the initial grain face got and
After reference line is expanded respectively, after being equal to the rectangle frame covering expansion of training set correspondence image width using a width respectively
Grain face and reference line, grain face and reference line are refined respectively using GrabCut algorithm or full connection CRF algorithm.
Further, in order to reduce identification error, grain face image is divided using Image Segmentation Model in obtained storehouse
After cutting, prior information using grain face in the lower section of line of placing rice, the region that wrong point of removal.
Further, the reference line is line of placing rice, after refining to the reference line, obtains the top of line of placing rice
Boundary.With the coboundary of line of placing rice be with reference to can prevent line of placing rice part by grain cover and can not accurate measurements grain face change
Change.
Further, it in order to improve the calculating accuracy of the area between grain face and reference line, realizes described in the calculating
The process of the area between grain face and reference line in grain face image includes: to calculate each pixel of grain face to line of placing rice
The area between grain face and the coboundary of line of placing rice is calculated according to each vertical range in vertical range between coboundary.
The silo Grain Quantity monitoring device based on convolutional neural networks that the present invention also provides a kind of, including memory, place
In the memory and the computer program that can run on a processor, the processor is described in the execution for reason device and storage
The treatment process realized when computer program includes:
(1) grain face image is acquired；The grain face image includes the reference line above grain face and grain face；
(2) the grain face image is input in the storehouse based on convolutional neural networks put up in Image Segmentation Model,
Identify grain face and reference line；
(3) grain face in the grain face image and the area between reference line are calculated, and is calculated between preset area
Error, obtain error amount, if the error amount be greater than setting error threshold, determine that silo Grain Quantity changes.
This programme carries out oneself of silo Grain Quantity variation using Image Segmentation Model in the storehouse based on convolutional neural networks
Dynamic monitoring, is handled collected grain face image by the way of image procossing, final process obtains silo Grain Quantity
Whether change, the corresponding monitoring method of the device is one and combines the image based on convolutional neural networks using grain face image
Processing technique carries out the automatic monitoring method of automatic monitoring Grain Quantity variation, realizes Grain Quantity peace based on computer vision
Full monitoring function.Relative to existing method, this method does not need to put into too many professional equipment, it is only necessary to put into Image Acquisition
With processing equipment, then carrying out processing to image can be realized as monitoring automatically, and at low cost and deployment is convenient.Also, only
It need to detecting grain face image, by just can determine whether silo Grain Quantity changes after processing, there is detection speed
Fastly, the advantages of low energy consumption.Therefore, the monitoring accuracy of the corresponding monitoring method of the device and reliability are higher, can be effective
Push grain depot intelligentized updating that process is transformed.Furthermore it is also possible in conjunction with the detection method scanned based on infrared laser, when discovery grain
When face changes, that is, it infrared laser scanner can be used to carry out whole storehouse scanning, obtain accurate grain volume, to delay infrared sharp
The service life of light.
Further, in order to improve silo Grain Quantity monitoring reliability, Image Segmentation Model was built in the storehouse
Journey include: acquisition grain face sample image, in grain face sample image grain face and reference line be marked, generate training set；
Convolutional neural networks model is built, training set is input to convolutional neural networks model and is trained, image segmentation in storehouse is obtained
Model.
Detailed description of the invention
Fig. 1 is the monitoring framework signal of the silo Grain Quantity monitoring method provided by the invention based on convolutional neural networks
Figure；
Fig. 2 is the monitoring flow chart of the silo Grain Quantity monitoring method provided by the invention based on convolutional neural networks；
Fig. 3 is the areal calculation schematic diagram between the fitting of line of placing rice coboundary and grain face；
Wherein, it is 1. line of placing rice, is 2. grain face, is 3. grain outlet.
Specific embodiment
The present invention will be further described in detail with reference to the accompanying drawing.
Fig. 1 is the monitoring framework signal of the silo Grain Quantity monitoring method provided by the invention based on convolutional neural networks
Figure, as shown in Figure 1, include video camera C, line of placing rice 1., grain face 2. with grain outlet 3..1. line of placing rice is distributed in silo surrounding.Its
In, video camera C is used to obtain grain outlet of granary and 3. locates to include the line of placing rice 1. image with grain face 2..Certainly, as other real
Mode is applied, line of placing rice could alternatively be other reference lines (or referred to as marker), such as: it could alternatively be in line of placing rice
The mark line of either above or below setting, is not much different, alternatively, it is also possible to according to monitoring under normal circumstances at a distance from line of placing rice
It needs, replaces with the mark line in the setting of other corresponding positions.It is according to mark line, then the line of placing rice in subsequent narration is whole
Mark line is replaced with, method, the step being related to are consistent.Wherein, line of placing rice and mark line belong to reference line, use
In calculate grain face whole height when reference is provided.
To make video camera shoot image in designated position every time, camera pan-tilt parameter is set, video camera shooting is preset
The parameters such as horizontal angle, vertical angle, aperture make to guarantee that captured image covers as far as possible at video camera alignment grain outlet and put out cereal
The grain face information of mouth, needs to be arranged four sets of parameters if detecting four grain outlets, while also it is contemplated that captured image covering
Entire silo.In order to automate whole process, at the same consider when grain face be buy securities with all one's capital, half storehouse and when hole capital after selling all securities the position of video camera and
The parameters such as aperture enable the system to the different parameters for calling camera preset automatically according to current detection result, use backstage Xiang Yun
Platform sends order and completes parameter transmitting and execute relevant operation, and the parameters such as detection interval, start by set date is arranged in background system
Video camera simultaneously calls corresponding preset parameter to capture grain face image in storehouse.In the present embodiment, for convenience of description, monitoring one is put out cereal
Mouthful, therefore only need that a set of camera parameters are arranged, then, in the present embodiment, become by only monitoring the grain face at a grain outlet
Change situation to judge whether quantity of stored grains in granary changes；As other embodiments, can also by monitoring silo in it is multiple go out
Grain face situation of change at grain mouth judges whether quantity of stored grains in granary changes.In addition, though above-mentioned monitored at grain outlet
Grain face variation, still, the mentioned method of the present invention are not limited to the progress grain face monitoring at grain outlet, can also in storehouse other positions
It sets region and carries out grain face monitoring.
Generally speaking, the method for the present invention is intended to by monitoring grain automatically by means of camera in storehouse on the basis of line of placing rice
Whether face changes relative to line of placing rice, to judge whether grain occurs unusual fluctuation in storehouse.If desired to close to the whole of wall
A grain face is monitored, then needs to obtain image in one week storehouse.When obtaining image at grain outlet, the level of video camera is set
The parameters such as angle, vertical angle and aperture, so that video camera can shoot the image in same place every time.
Based on monitoring framework shown in FIG. 1, following detailed processes for providing silo Grain Quantity monitoring method, such as Fig. 2 institute
Show, certainly, the invention is not limited to monitoring frameworks shown in FIG. 1.
For buying securities with all one's capital (half storehouse and hole capital after selling all securities can be realized by the way that multiple presetting bits are arranged for camera), obtains put out cereal in advance
Several images (i.e. grain face sample image) comprising line of placing rice and grain face are used as training image collection at mouthful, using labelme tool
Training image is labeled, training image is labeled as 4 classes: grain face, line of placing rice, window and background save training image
Annotation results (i.e. mark image), i.e., be divided into four classes for the pixel in all training images, it may be assumed that line of placing rice, grain face, window and its
Its background information.Certainly, it is above-mentioned be a kind of optimization notation methods, since method provided by the invention is according to grain face and loading
Line realizes that Grain Quantity monitors, and therefore, as general embodiment, only need to mark grain face and line of placing rice can be realized as grain
Eat Monitoring of Quantity.
The training image collection marked is enhanced, the main region of interest that specified size is intercepted using a fixed step size
Domain, adjustment image Gamma correction parameter, zoomed image, flipped image, rotation are no more than ± 10 degree relative to original image, increase
The operation such as Gaussian noise is added to enhance training image collection, generation meets successive depths convolutional neural networks model training
The training image collection needed.Original training image and mark image change simultaneously during enhancing, if mark image has interpolation behaviour
Elect arest neighbors interpolation.Because grain face and line of placing rice have certain semantic context relationship, cannot to original training image into
The rotation of row wide-angle.Scheduled target training image collection number is set, by taking target training image number is 5000 as an example, according to target
The mode of number setting image enhancement.For any one training image and its tag image, with one since the upper left corner of image
Fixed step size intercepts the area-of-interest of specified size, is inverted respectively to the region, the adjustment of Gamma parameter, is no more than ± 10
Operation, each operations such as rotation, the addition Gaussian noise of degree have corresponding parameter sets, ultimately generate close to number of targets
The training image collection of amount.Training sample set is capable of increasing by using the method that image set enhances in training process, that is, increases instruction
Practice the scale of sample set.It is above-mentioned to give a kind of specific image enhancement processes, but the invention is not limited to above-mentioned specific
Process can select suitable image enhancement mode according to actual needs, can be with alternatively, if training image collection is met the requirements
Training image is not enhanced.
Then, convolutional neural networks model is built, training image collection is input to convolutional neural networks model and is trained,
Obtain Image Segmentation Model in storehouse.Specific as follows: then selected depth neural network framework first is arranged required for training process
Parameter, training image collection inputs mould under GPU environment by the value including learning rate, parameter loss ratio, epoch and batch etc.
Type is trained, and saves trained deep neural network model, in case used in test.The full convolutional neural networks model built
The models such as deeplab, segNet and U-Net can be used.By taking deeplab v2 model as an example, depth is realized using caffe frame
The training of full convolutional neural networks model.Specific step is as follows:
Firstly, generating the ID file train_id.txt of training image and the respective path text of training image and annotation results
Part train.txt；
Then, train.prototxt file required for model training is set, will wherein crop_size parameter be changed to
417, the mean value of training image is set, and model output is 4 classes etc.；Use vgg16 neural network model for basic model, according to
The requirement of deeplab v2 model increases corresponding up-sampling layer；Original model parameter is instructed on voc2012 database using VGG16
Experienced result；
Then, using cross entropy, the loss function of marginal loss building model；
Further, in the training process, learning rate is set for 1e-3, learning rate drawdown parameter 0.9, average every 20 calculating
Primary loss, maximum number of iterations 20000, parameter attenuation rate 0.0005；
Further, training image input model carries out forward calculation, obtains prediction knot by the softmax of the last layer
Fruit calculates loss function, while the parameter declined according to the value in current network by gradient in conjunction with handmarking's result
Iterative formula undated parameter；
Finally, terminating training when network reaches maximum number of iterations or preset stop condition, obtaining image segmentation in storehouse
Model.
Certainly, the training and test that U-net and Tensorflow realizes the model also can be used in deep neural network framework,
Since U-Net is suitble to two classes to divide, the training of 3 two class parted patterns is carried out to grain face, line of placing rice and window respectively, is protected
Trained model is deposited, multiclass parted pattern training method can be used in other models.Therefore, of the invention to focus on establishing base
In Image Segmentation Model in the storehouse of convolutional neural networks frame, it is not limited to use which kind of full convolutional neural networks frame, root
Corresponding establishment process is carried out according to the full convolutional neural networks frame of selection.
In monitoring process, the true grain face image (i.e. test image) of a width at grain outlet, certainly, the grain are obtained first
It include grain face and line of placing rice in the image of face.It is training image size by the grain face image scaling, and is input to trained storehouse
In interior Image Segmentation Model, the grain face image is split using Image Segmentation Model in trained storehouse, is obtained initial
Line of placing rice, grain face and window etc..Further, Image Segmentation Model only carries out forward calculation and is predicted in trained storehouse
Value, to obtain segmentation result, further, is scaled original image size for segmentation result.
After acquiring initial line of placing rice, grain face and window, band-like prior information is had according to line of placing rice and grain face,
I.e. grain face must be in line of placing rice hereinafter, window must remove some wrong point of regions, i.e. noise region in line of placing rice with first-class.
Boundary using the obtained initial line of placing rice of Image Segmentation Model in storehouse and grain face is relatively rough, needs to line of placing rice
It is refined with grain face, the specific steps are as follows: first line of placing rice and grain face are expanded, then utilize Image Segmentation Model in storehouse
The pixel coordinate information of middle line of placing rice and grain face covers line of placing rice using the rectangle frame that a width is training image width respectively
And grain face, line of placing rice and grain face are refined respectively on pixel level using GrabCut algorithm.Loading is removed when refining grain face
Region more than line generates a rectangle frame bigger comprising line of placing rice again, big rectangle frame is regarded as when refining line of placing rice
For image, line of placing rice is refined, obtains final segmentation result, is believed by the boundary that this step corrects line of placing rice and grain face
Breath.Specifically, for line of placing rice: first carrying out expansion process to line of placing rice, using a rectangle frame, width is same as test chart
Picture, the line of placing rice after covering expansion, as priori knowledge, i.e. line of placing rice one is scheduled in frame, then, the structure again other than the rectangle frame
A rectangle frame is built, guarantees that the number of pixels in line of placing rice and other regions is balanced in outer layer rectangle frame, is then foreground and background
Gauss hybrid models are established, data item uses the probability of deep neural network model output in energy term, and smooth Xiang Ze is used
RGB the and Lab color characteristic of pixel calculates, and is refined using GrabCut algorithm to line of placing rice, obtains final line of placing rice
Segmentation result identifies the coboundary of line of placing rice, be that reference can prevent line of placing rice part by grain with the coboundary of line of placing rice
Food cover and can not accurate measurements grain face variation.Similarly, simultaneously using the grain face in identical method process processing segmentation result
Obtain its final segmentation result.Among the above, it is not restricted to using GrabCut algorithm, other such as connection CRF algorithm can also entirely
For the refinement to image segmentation result.
Obtain line of placing rice coboundary pixel coordinate information, with prevent grain cover line of placing rice the case where.Utilize segmentation
Least square method is fitted line of placing rice coboundary straight line, replaces line of placing rice coboundary with fitting a straight line, to remove exterior point and noise spot,
Segmentation number is up to 2 herein, the straight line of silo one side and corner is simulated with this, and as line of placing rice coboundary.Certainly,
Above-mentioned straight line fitting is a kind of embodiment of optimization, can also be without straight line fitting as other embodiments.So
Afterwards, the pixel of the coboundary of each line of placing rice is calculated to the vertical range between grain face (i.e. grain face coboundary), alternatively, meter
Each pixel on grain face coboundary is calculated to the vertical range between line of placing rice coboundary, as shown in figure 3, calculating loading accordingly
The distance between line and grain face, since the variation of the area between line of placing rice and grain face changes more than average distance between the two
Add sensitivity, for this purpose, according to each pixel of grain face to the vertical range between line of placing rice coboundary calculate line of placing rice and grain face it
Between region area, i.e., by integral in the way of zoning area.
Assuming that current line of placing rice to the region area between grain face be Ac, set a preset area, the preset area with
The region area A of historical recordhFor, calculate AcWith AhDifference R, i.e. R=| Ac-Ah|, if R is greater than the error threshold of setting,
Think that biggish variation has occurred in grain face, i.e. judgement silo Grain Quantity changes, and issues and alerts to administrator, otherwise records
Current data.
In order to verify the superiority of method provided by the invention, a series of experiment of quantitative analyses is devised.
Image in 10 width storehouses is acquired, the size of each image is that 1080 × 1920,10 images are in different illumination, preset
It is shot under angle.Wherein 6 width images are training image collection, carry out handmarking to training image collection and obtain handmarking's result
(Ground Truth), so that the precision to segmentation result is evaluated.
Training image is labeled to and is passed through image set enhancing, obtains mark image.
Mark image scaling (is scaled 512 × 512 and is used for U- to 417 × 417 for deeplab v2 model training
Net model training), obtain trained deep neural network model.
By image scaling to be split to 417 × 417 and it is input to trained deep neural network model progress image point
It cuts, and obtained segmentation result is scaled into back original image size.Then, the more than line of placing rice wrong region for being divided into grain face is reset to
The region for being divided into window wrong below line of placing rice is reset to grain face or background according to the similarity of neighborhood territory pixel by background, thus
Segmentation result to the end.
When being refined using GrabCut to line of placing rice and grain face, the weight of smooth item is selected as 50.
Final segmentation result is tested first.After obtaining final segmentation result, using four kinds of image segmentation fields
Evaluation criterion evaluates these three segmentation results.As shown in table 1, it is respectively as follows: PRI (Probabilistic Rand
Index, probability Rand index), VOI (Variation of Information, information change index), GCE (global coherency
Error extension) and BDE (boundary shifts error extension)；Wherein, the size of PRI is in section [0,1], the bigger expression segmentation of value
Accuracy it is higher；The size of VOI [0 ,+∞) in section, the smaller effect for indicating segmentation of value is better；BDE, GCE's takes
It is better to be worth smaller expression segmentation effect, it is as a result closer with Ground Truth.
Table 1
| Method | PRI | VOI | GCE | BDE |
| Ground Truth | 0.98 | 1.20 | 0.11 | 0.56 |
| The method of the present invention | 0.97 | 1.22 | 0.12 | 0.58 |
From table 1 it follows that the method segmentation result of the invention very close artificial mark in four evaluation indexes
Note is as a result, therefore can illustrate the validity of the method for the present invention.
In grain face variation monitoring, the video camera with holder is fixed on wall, relevant parameter is set, away from camera shooting
A liftable plank is placed in the position that 15 meters of machine, the wheat with a thickness of 2 centimetres is uniformly placed above plank, and use one
Baffle of the white with red mark line simulates storehouse interior walls, simulates grain face variation by the plank that grain is placed in lifting.Pass through
It tests three times, each grain face declines about 12 centimetres, and testing result is as shown in table 2.
Table 2
| Pendulous frequency | Grain face height (rice) | Area (pixel2) | Difference in areas |
| 1 | 0.385 | 122737 | |
| 2 | 0.26 | 167808 | 45071 |
| 3 | 0.14 | 214584 | 46776 |
From Table 2, it can be seen that grain face height is obtained the case where grain face declines about 0.12 meter respectively three times in test
, difference in height is the difference between second and first time respectively, the difference between third time and second, and average distance is
It is calculated according to picture traverse is 640 pixels, the adjacent area difference measured twice is 45071 and 46776 respectively, usually
In the case of, it is 39000 that threshold value is arranged herein, calculates that 39000 are approximately equal to 0.1 meter of actual height according to experiment.Experimental result
Show that practical grain face decline corresponds to the variation of number of pixels, can satisfy the demand of Grain Quantity variation monitoring, for this purpose, card
Bright the method for the present invention is feasible.
Specific embodiment is presented above, but the present invention is not limited to described embodiment.Base of the invention
This thinking is above-mentioned basic scheme, and for those of ordinary skill in the art, various changes are designed in introduction according to the present invention
The model of shape, formula, parameter do not need to spend creative work.It is right without departing from the principles and spirit of the present invention
The change, modification, replacement and modification that embodiment carries out are still fallen in protection scope of the present invention.
The above-mentioned silo Grain Quantity monitoring method based on convolutional neural networks can be used as a kind of computer program, storage
In the memory of the silo Grain Quantity monitoring device based on convolutional neural networks, and can be by the grain based on convolutional neural networks
The processor of storehouse Grain Quantity monitoring device executes.
Claims (10)
1. a kind of silo Grain Quantity monitoring method based on convolutional neural networks, which comprises the following steps:
(1) grain face image is acquired；The grain face image includes the reference line above grain face and grain face；
(2) the grain face image is input in the storehouse based on convolutional neural networks put up in Image Segmentation Model, is identified
Grain face and reference line out；
(3) grain face in the grain face image and the area between reference line are calculated, and calculates the mistake between preset area
Difference obtains error amount, if the error amount is greater than the error threshold of setting, determines that silo Grain Quantity changes.
2. the silo Grain Quantity monitoring method according to claim 1 based on convolutional neural networks, which is characterized in that institute
State Image Segmentation Model in storehouse build process include: acquisition grain face sample image, in grain face sample image grain face and
Reference line is marked, and generates training set；Convolutional neural networks model is built, training set is input to convolutional neural networks model
It is trained, obtains Image Segmentation Model in storehouse.
3. the silo Grain Quantity monitoring method according to claim 2 based on convolutional neural networks, which is characterized in that institute
It states in storehouse in the build process of Image Segmentation Model, to the grain face in grain face sample image and after reference line is marked,
More massive training set is generated by the way of image set enhancing.
4. the silo Grain Quantity monitoring method according to claim 2 or 3 based on convolutional neural networks, feature exist
In, in step (2), identify grain face and reference line in grain face image process include: by grain face image scaling to instruction
Practice the size for collecting corresponding image, grain face image is split using Image Segmentation Model in obtained storehouse, is obtained initial
Grain face and reference line；Then, the initial grain face and reference line got is expanded respectively；Finally, to grain face with
And reference line is refined respectively, identification obtains grain face and reference line.
5. the silo Grain Quantity monitoring method according to claim 4 based on convolutional neural networks, which is characterized in that right
After the initial grain face and reference line got is expanded respectively, it is equal to training set correspondence image using a width respectively
Grain face and reference line after the rectangle frame covering expansion of width are right respectively using GrabCut algorithm or full connection CRF algorithm
Grain face and reference line are refined.
6. the silo Grain Quantity monitoring method according to claim 4 based on convolutional neural networks, which is characterized in that make
After being split with Image Segmentation Model in obtained storehouse to grain face image, the priori using grain face in the lower section of line of placing rice is believed
Breath, the region that wrong point of removal.
7. the silo Grain Quantity monitoring method according to claim 5 based on convolutional neural networks, which is characterized in that institute
Stating reference line is line of placing rice, after refining to the reference line, obtains the coboundary of line of placing rice.
8. the silo Grain Quantity monitoring method according to claim 7 based on convolutional neural networks, which is characterized in that real
The process of area between the existing grain face calculated in the grain face image and reference line includes: each picture for calculating grain face
The top of grain face and line of placing rice is calculated according to each vertical range to the vertical range between the coboundary of line of placing rice for vegetarian refreshments
Area between boundary.
9. a kind of silo Grain Quantity monitoring device based on convolutional neural networks, including memory, processor and it is stored in
In the memory and the computer program that can run on a processor, which is characterized in that the processor is executing the meter
The treatment process realized when calculation machine program includes:
(1) grain face image is acquired；The grain face image includes the reference line above grain face and grain face；
(2) the grain face image is input in the storehouse based on convolutional neural networks put up in Image Segmentation Model, is identified
Grain face and reference line out；
(3) grain face in the grain face image and the area between reference line are calculated, and calculates the mistake between preset area
Difference obtains error amount, if the error amount is greater than the error threshold of setting, determines that silo Grain Quantity changes.
10. the silo Grain Quantity monitoring device according to claim 9 based on convolutional neural networks, which is characterized in that
In the storehouse build process of Image Segmentation Model include: acquisition grain face sample image, to the grain face in grain face sample image with
And reference line is marked, and generates training set；Convolutional neural networks model is built, training set is input to convolutional neural networks mould
Type is trained, and obtains Image Segmentation Model in storehouse.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910295466.1A CN110008947B (en) | 2019-04-12 | 2019-04-12 | Granary grain quantity monitoring method and device based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910295466.1A CN110008947B (en) | 2019-04-12 | 2019-04-12 | Granary grain quantity monitoring method and device based on convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008947A true CN110008947A (en) | 2019-07-12 |
| CN110008947B CN110008947B (en) | 2021-06-29 |
Family
ID=67171558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910295466.1A Active CN110008947B (en) | 2019-04-12 | 2019-04-12 | Granary grain quantity monitoring method and device based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008947B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111200723A (en) * | 2020-01-07 | 2020-05-26 | 苏州恒志汇智能科技有限公司 | Progressive die arch material monitoring method, device, equipment and storage medium |
| CN111507148A (en) * | 2019-12-31 | 2020-08-07 | 浙江苏泊尔家电制造有限公司 | Control system and control method of rice storage device |
| CN111582778A (en) * | 2020-04-17 | 2020-08-25 | 上海中通吉网络技术有限公司 | Operation site cargo accumulation measuring method, device, equipment and storage medium |
| CN111967473A (en) * | 2020-06-29 | 2020-11-20 | 山东浪潮通软信息科技有限公司 | Grain depot storage condition monitoring method based on image segmentation and template matching |
| CN112734826A (en) * | 2020-12-29 | 2021-04-30 | 华信咨询设计研究院有限公司 | Grain quantity estimation method based on deep learning and LSD (least squares-based) linear detection algorithm |
| CN116755088A (en) * | 2023-08-09 | 2023-09-15 | 中国科学院空天信息创新研究院 | Granary depth and foreign matter detection and imaging method based on radar |
| EP4273517A1 (en) * | 2022-05-06 | 2023-11-08 | Insylo Technologies, S.L. | Method and system for measuring the quantity of a bulk product stored in a silo |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103063136A (en) * | 2012-12-28 | 2013-04-24 | 大连工大（泗阳）光源与照明工程技术研究院有限公司 | Detecting system for granary reserves |
| CN105674908A (en) * | 2015-12-29 | 2016-06-15 | 中国科学院遥感与数字地球研究所 | Measuring device, and volume measuring and monitoring system |
| US20170119658A1 (en) * | 2015-11-01 | 2017-05-04 | Justin Turvey | Preparation for enhanced fingerprint image formation on a transparent surface of a live scan device |
| US20170173262A1 (en) * | 2017-03-01 | 2017-06-22 | François Paul VELTZ | Medical systems, devices and methods |
| CN108090628A (en) * | 2018-01-16 | 2018-05-29 | 浙江大学 | A kind of grain feelings security detection and analysis method based on PSO-LSSVM algorithms |
| CN108833313A (en) * | 2018-07-12 | 2018-11-16 | 北京邮电大学 | A kind of radio channel estimation method and device based on convolutional neural networks |
| CN109472261A (en) * | 2018-06-15 | 2019-03-15 | 河南工业大学 | A kind of quantity of stored grains in granary variation automatic monitoring method based on computer vision |
-
2019
- 2019-04-12 CN CN201910295466.1A patent/CN110008947B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103063136A (en) * | 2012-12-28 | 2013-04-24 | 大连工大（泗阳）光源与照明工程技术研究院有限公司 | Detecting system for granary reserves |
| US20170119658A1 (en) * | 2015-11-01 | 2017-05-04 | Justin Turvey | Preparation for enhanced fingerprint image formation on a transparent surface of a live scan device |
| CN105674908A (en) * | 2015-12-29 | 2016-06-15 | 中国科学院遥感与数字地球研究所 | Measuring device, and volume measuring and monitoring system |
| US20170173262A1 (en) * | 2017-03-01 | 2017-06-22 | François Paul VELTZ | Medical systems, devices and methods |
| CN108090628A (en) * | 2018-01-16 | 2018-05-29 | 浙江大学 | A kind of grain feelings security detection and analysis method based on PSO-LSSVM algorithms |
| CN109472261A (en) * | 2018-06-15 | 2019-03-15 | 河南工业大学 | A kind of quantity of stored grains in granary variation automatic monitoring method based on computer vision |
| CN108833313A (en) * | 2018-07-12 | 2018-11-16 | 北京邮电大学 | A kind of radio channel estimation method and device based on convolutional neural networks |
Non-Patent Citations (2)
| Title |
|---|
| ZHANG XIAODONG 等: "Environmental Monitoring in Grain Granary Based on Embedded System", 《IEEE》 * |
| 李磊 等: "基于自适应区域限制FCM的图像分割方法", 《电子学报》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111507148A (en) * | 2019-12-31 | 2020-08-07 | 浙江苏泊尔家电制造有限公司 | Control system and control method of rice storage device |
| CN111507148B (en) * | 2019-12-31 | 2023-10-24 | 浙江苏泊尔家电制造有限公司 | Control system and control method of rice storage device |
| CN111200723A (en) * | 2020-01-07 | 2020-05-26 | 苏州恒志汇智能科技有限公司 | Progressive die arch material monitoring method, device, equipment and storage medium |
| CN111582778A (en) * | 2020-04-17 | 2020-08-25 | 上海中通吉网络技术有限公司 | Operation site cargo accumulation measuring method, device, equipment and storage medium |
| CN111582778B (en) * | 2020-04-17 | 2024-04-12 | 上海中通吉网络技术有限公司 | Method, device, equipment and storage medium for measuring accumulation of cargos in operation site |
| CN111967473A (en) * | 2020-06-29 | 2020-11-20 | 山东浪潮通软信息科技有限公司 | Grain depot storage condition monitoring method based on image segmentation and template matching |
| CN112734826A (en) * | 2020-12-29 | 2021-04-30 | 华信咨询设计研究院有限公司 | Grain quantity estimation method based on deep learning and LSD (least squares-based) linear detection algorithm |
| CN112734826B (en) * | 2020-12-29 | 2022-05-31 | 华信咨询设计研究院有限公司 | Grain quantity estimation method based on deep learning and LSD (least squares-based) linear detection algorithm |
| EP4273517A1 (en) * | 2022-05-06 | 2023-11-08 | Insylo Technologies, S.L. | Method and system for measuring the quantity of a bulk product stored in a silo |
| WO2023213863A1 (en) * | 2022-05-06 | 2023-11-09 | Insylo Technologies, S.L. | Method and system for measuring the quantity of a bulk product stored in a silo |
| CN116755088A (en) * | 2023-08-09 | 2023-09-15 | 中国科学院空天信息创新研究院 | Granary depth and foreign matter detection and imaging method based on radar |
| CN116755088B (en) * | 2023-08-09 | 2023-11-17 | 中国科学院空天信息创新研究院 | Granary depth and foreign matter detection and imaging method based on radar |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008947B (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110008947A (en) | A kind of silo Grain Quantity monitoring method and device based on convolutional neural networks | |
| CN110287932B (en) | Road blocking information extraction method based on deep learning image semantic segmentation | |
| CN111353413B (en) | Low-missing-report-rate defect identification method for power transmission equipment | |
| CN108257121B (en) | Method, apparatus, storage medium and the terminal device that product defects detection model updates | |
| CN108460760B (en) | Bridge crack image distinguishing and repairing method based on generation type countermeasure network | |
| US8577611B2 (en) | System and method for analyzing trees in LiDAR data using views | |
| US20130155235A1 (en) | Image processing method | |
| CN109613002A (en) | A kind of glass defect detection method, apparatus and storage medium | |
| CN107451999A (en) | foreign matter detecting method and device based on image recognition | |
| CN109472261A (en) | A kind of quantity of stored grains in granary variation automatic monitoring method based on computer vision | |
| CN108615046A (en) | A kind of stored-grain pests detection recognition methods and device | |
| CN110648322A (en) | Method and system for detecting abnormal cervical cells | |
| CN109253722A (en) | Merge monocular range-measurement system, method, equipment and the storage medium of semantic segmentation | |
| CN110378227B (en) | Method, device and equipment for correcting sample labeling data and storage medium | |
| CN111079518B (en) | Ground-falling abnormal behavior identification method based on law enforcement and case handling area scene | |
| CN109711322A (en) | A kind of people's vehicle separation method based on RFCN | |
| CN108090493A (en) | It is a kind of based on wifi positioning into shop customer data statistical method | |
| CN110264444A (en) | Damage detecting method and device based on weak segmentation | |
| CN107218894A (en) | A kind of subpixel accuracy thickness of detector detection method of fast and stable | |
| CN113435282A (en) | Unmanned aerial vehicle image ear recognition method based on deep learning | |
| US20220128358A1 (en) | Smart Sensor Based System and Method for Automatic Measurement of Water Level and Water Flow Velocity and Prediction | |
| CN106339710B (en) | A kind of border width defect inspection method and device | |
| CN115049598A (en) | Method, system and equipment for detecting standard of trial product placed on store shelf | |
| US20170048518A1 (en) | Method and apparatus for adjusting installation flatness of lens in real time | |
| CN112750104A (en) | Method and device for automatically matching optimal camera by monitoring ship through multiple cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |