CN109472261B - Computer vision-based automatic monitoring method for grain storage quantity change of granary - Google Patents
Computer vision-based automatic monitoring method for grain storage quantity change of granary Download PDFInfo
- Publication number
- CN109472261B CN109472261B CN201810621000.1A CN201810621000A CN109472261B CN 109472261 B CN109472261 B CN 109472261B CN 201810621000 A CN201810621000 A CN 201810621000A CN 109472261 B CN109472261 B CN 109472261B
- Authority
- CN
- China
- Prior art keywords
- grain
- grain surface
- reference line
- image
- granary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Development Economics (AREA)
- Operations Research (AREA)
- Accounting & Taxation (AREA)
- Multimedia (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Finance (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a computer vision-based automatic monitoring method for grain storage quantity change of a granary, and belongs to the technical field of granary detection. Firstly, collecting a grain surface image at a grain outlet; the grain surface image comprises a reference line above the grain surface of the grain outlet, and the reference line is a grain loading line or a marking line; calculating the overall height of the grain by processing the grain image, and comparing the overall height of the grain with the overall height of the grain calculated at one time or the previous time; if the whole height of the grain surface is reduced, the number of stored grains in the bin is judged to be reduced. Compared with the prior art that the grain quantity in the whole granary needs to be detected continuously, the method can judge whether the grain storage quantity of the granary changes or not by detecting the grain surface information at the grain outlet and comparing the calculated integral height of the grain surface with the preset calculation result of one time or the previous time, and has the advantages of high detection speed and low energy consumption.
Description
Technical Field
The invention relates to a computer vision-based automatic monitoring method for grain storage quantity change of a granary, and belongs to the technical field of granary detection.
Background
In recent years, along with the intelligent construction of grain depots, in order to accurately master and analyze grain conditions in the grain depots, the currently commonly used technologies mainly include a pressure sensor-based stored grain quantity detection method, an infrared laser scanning-based stored grain quantity detection method and a ground penetrating radar-based stored grain quantity detection method. Compared with the traditional manual warehouse clearing and checking, the methods improve the efficiency and the detection precision of warehouse clearing and checking, but still have some defects, such as: the method based on the pressure sensor has higher requirement on the layout of the sensor, the cost of the sensor is higher, and the sensitivity of the sensor can be degraded along with time to reduce the detection precision; the infrared laser scanning-based method is limited by the service life of an infrared laser, so that the infrared laser scanning-based method cannot be used for a long time, and the time overhead of accurately scanning the granary once is relatively high; the ground penetrating radar-based method has high equipment installation and maintenance cost and single function, and is difficult to popularize.
In the prior art, a patent document with the publication number of CN 103063136B and the name of "a granary storage amount detection system" also discloses a method for calculating the volume storage amount of grains in a granary by setting black and white grid marks in the granary and adopting an image acquisition and analysis technology. The image processing mode can be used for overcoming the advantage that the granary is generally provided with the camera, and the cost input is not increased.
In summary, the prior art has the following disadvantages:
firstly, in the existing mode, in order to monitor the change of the grain quantity, a method for directly detecting the grain quantity in the whole granary is adopted. The method needs to continuously detect the grain quantity in the whole granary, and the consumed energy and time cost are high.
Although the image processing method is superior, the method in the above patent document has low detection accuracy; and need brush square sign on the granary inner wall, work load is big, and along with the increase of granary life, these signs are easily become fuzzy by wearing and tearing, can lead to the testing result inaccurate, need constantly carry out the later maintenance.
Disclosure of Invention
The invention aims to solve the problems of high energy consumption and time cost caused by the fact that the grain quantity in the whole granary needs to be continuously detected in the existing grain storage quantity monitoring method.
In order to solve the technical problem, the invention provides a computer vision-based automatic monitoring method for the quantity change of stored grains in a granary, which comprises the following steps:
(1) collecting grain surface images at a grain outlet; the grain surface image comprises a reference line above the grain surface of the grain outlet, and the reference line is a grain loading line or a marking line;
(2) calculating the overall height of the grain by processing the grain image, and comparing the overall height of the grain with the overall height of the grain calculated at one time or the previous time;
(3) if the whole height of the grain surface is reduced, the number of stored grains in the bin is judged to be reduced.
The invention has the beneficial effects that: compared with the prior art that the grain quantity in the whole granary needs to be detected continuously, the method can judge whether the grain storage quantity of the granary changes or not by detecting the grain surface information at the grain outlet and comparing the calculated integral height of the grain surface with the preset calculation result of one time or the previous time, and has the advantages of high detection speed and low energy consumption.
The method is characterized in that the overall height of the grain surface is calculated quantitatively, and is used as an improvement of the automatic monitoring method for the grain storage quantity change of the granary based on computer vision, wherein the overall height of the grain surface refers to the average distance between a reference line and the grain surface, and the average distance is obtained by calculating the average value of the vertical distance from each point of the upper boundary of the grain surface to the lower boundary of the reference line; the overall height reduction of the grain surface means that the average distance between the currently calculated reference line and the grain surface is larger than a preset or previous calculation result.
In order to further improve the calculation accuracy of the overall height of the grain surface, the method is used as another improvement of the automatic monitoring method for the grain storage quantity change of the granary based on computer vision, wherein the overall height of the grain surface refers to the area between a reference line and the grain surface, and the area is obtained in an integral mode after the vertical distance from each point of the upper boundary of the grain surface to the lower boundary of the reference line is calculated; the integral height reduction of the grain surface means that the area of the area between the currently calculated reference line and the grain surface is larger than the preset calculation result or the previous calculation result.
As a further improvement of the above automatic monitoring method for grain storage quantity change in a granary based on computer vision, in order to accurately segment a reference line and a grain surface of a grain surface image in real time, the processing of the grain surface image includes: extracting the characteristics of all sample images; training an SVM classifier model according to the extracted sample image characteristics; and then, segmenting the grain surface image to be processed by utilizing the trained SVM classifier model to obtain a reference line and grain surfaces.
In order to improve the accuracy of the segmentation result and improve the segmentation efficiency, as a further improvement of the automatic monitoring method for the grain storage quantity change of the granary based on computer vision, the process of segmenting the grain surface image to be processed comprises the following steps: preprocessing an image; generating high-level superpixels and bottom-level superpixels of the image; extracting bottom layer super pixel characteristics of the image; the process of extracting the bottom layer super pixel characteristics of the image comprises the following steps: when extracting the characteristics of the bottom layer superpixel, taking the high layer superpixel as space limitation, if the neighborhood pixels of one bottom layer superpixel are all under the same high layer superpixel, the final characteristics of the bottom layer superpixel are the weighted average of the characteristics of the neighborhood superpixel; otherwise, the color mean of the pixels within the underlying superpixel is used as its final feature.
In order to make the segmentation result more accurate, as a further improvement of the above-mentioned automatic monitoring method for the change of the grain quantity stored in the granary based on computer vision, the process of obtaining the reference line and the grain surface further includes: denoising, refining and fitting the lower boundary of the reference line.
As a further improvement to the above automatic monitoring method for grain quantity change in a granary based on computer vision, the denoising process comprises: and removing small hollow areas and areas with the width less than half of the image width in the segmentation result by using the prior information that the reference line and the grain surface are in a belt shape, and merging the small hollow areas and the areas with the width less than half of the image width into the most similar adjacent areas according to the neighborhood similarity to obtain a further segmentation result.
As a further improvement of the above automatic monitoring method for the change of the grain quantity in the granary based on computer vision, the refining process comprises the following steps: and respectively expanding the reference line and the grain surface in the further segmentation result, then respectively covering the reference line and the grain surface by using a rectangular frame with the width being the image width, and respectively thinning the reference line and the grain surface by a segmentation method based on graph theory to obtain a final segmentation result.
As a further improvement of the above automatic monitoring method for grain storage quantity change in a granary based on computer vision, the process of fitting the lower boundary of the reference line comprises: and performing linear fitting on the lower boundary of the reference line in the final segmentation result, and taking the fitted straight line as the lower boundary of the reference line.
Drawings
FIG. 1 is a schematic view of a monitoring framework of an embodiment of the present invention;
FIG. 2 is a flow chart of an automatic detection of an embodiment of the present invention;
FIG. 3 is a schematic diagram of fitting of the lower boundary of the grain loading line and calculation of the overall height of the grain surface.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Example 1
FIG. 1 is a schematic view of a monitoring framework according to an embodiment of the present invention, as shown in the drawings: comprises a camera C, a grain loading line I, grain flour II and a grain outlet III; the grain loading lines are distributed around the granary. The camera C is used for acquiring images of a grain loading line I and grain surface II at a grain outlet of the granary.
In other embodiments, the grain loading line may be replaced with a marking line, for example: the mark line can be replaced by a mark line which is arranged 20-50 cm above the grain loading line; and marking lines arranged at other corresponding positions can be replaced according to the monitoring requirement. If the marking line is adopted, the grain loading line in the subsequent description is replaced by the marking line, and the related methods and steps are consistent. Wherein, the grain loading line and the marking line both belong to a reference line and are used for providing reference when calculating the overall height of the grain surface.
In order to make the camera aim at the position of the grain outlet, a square sparse grid mark similar to a calibration plate can be arranged at the position of the grain outlet of the granary; then presetting parameters such as a horizontal angle, a vertical angle, an aperture, a detection interval and the like shot by a camera when the grain level is full, half or empty, so that the camera can shoot images of the same place each time, and ensuring that the shot images cover the grain level information at a grain outlet as much as possible; the background system can automatically call different preset parameters of the camera, and completes parameter transmission and execution of related operations by sending commands to the pan-tilt.
In the embodiment, only one grain outlet is monitored, so that only one set of camera parameters is needed to be set; in other embodiments, multiple sets of camera parameters may be set if multiple outlets are to be monitored.
In the embodiment, whether the grain storage quantity of the granary changes or not is judged by monitoring the grain surface change condition at one grain outlet; in other embodiments, whether the grain storage quantity of the granary changes can be judged by monitoring the grain surface change conditions at a plurality of grain outlets in the granary.
Fig. 2 is a flow chart of automatic detection according to an embodiment of the present invention. Before the change of the grain storage quantity of the granary is automatically detected, the SVM classifier model needs to be trained firstly, and the method comprises the following specific steps:
1) acquiring a training set: acquiring a plurality of images containing grain loading lines and grain surfaces at a grain outlet, manually marking, and dividing each image into three types of grain loading lines, grain surfaces and other background information to be used as a training set;
2) image preprocessing: because the images may be shot under different illumination and preset angles, the images need to be preprocessed, namely distortion areas at the upper side, the lower side, the left side and the right side of each image in a training set are cut out, and the images are zoomed;
3) super-pixel generation: two kinds of superpixels of the image are generated by using mean shift and SLIC methods, wherein the area sizes of the superpixels generated by using the mean shift method are different, and the objects can be more accurately described to serve as high-level superpixels of the image; the super-pixels generated by the SLIC method are small in area and close in size and serve as bottom layer super-pixels of the image, and the bottom layer super-pixels are used for replacing pixels to serve as basic units for subsequent operation;
4) feature extraction and selection: for simplicity, only the CIELAB features of the image are used in extracting the underlying superpixel features. When extracting the characteristics of the bottom layer super-pixel, the high layer super-pixel is used as space limitation, if the neighborhood pixels of one bottom layer super-pixel are all under the same high layer super-pixel, the final characteristic of the bottom layer super-pixel is the neighborhood super-pixelA weighted average of the features of the elements; otherwise, using the color mean value of the pixels in the bottom layer super-pixel as the final characteristic; final feature F of ith underlying superpixeliComprises the following steps:
wherein, FiIs the final feature of the ith underlying superpixel,
neighborhood superpixels (i.e., neighborhood superpixels at a distance of 1 superpixel), F, representing the underlying superpixel ii oIs the average of the colors in the ith underlying superpixel, (. to) is the standard Kronecker's Delta function,/iAnd ljThe ith and jth bottom layer super-pixel belong to the labels of the high layer super-pixels, if the two belong to the same high layer super-pixel, then li＝＝lj。
Through the step, the bottom layer super-pixel not only contains neighborhood information of the pixel, but also contains neighborhood information of the super-pixel, the boundary of the image is kept while the region consistency is ensured, the operation efficiency is effectively improved, and the noise influence is greatly reduced.
5) Training an SVM classifier model: and training an SVM classifier model by using the final characteristics of the bottom layer superpixel obtained in the step, obtaining the trained SVM classifier model and storing the trained SVM classifier model.
Secondly, acquiring an image at a grain outlet in the monitoring process, performing image preprocessing, superpixel generation and feature extraction and selection operations, and segmenting the image by using a trained SVM classifier model to obtain an initial segmentation result of a grain loading line and a grain surface.
Then, segmenting and thinning the initial segmentation results of the grain loading line and the grain surface obtained by the SVM classifier model: firstly, removing small hollow areas and areas with the diameter smaller than half of the image width in the segmentation result by using prior information that a grain loading line and a grain surface are in a belt shape, merging the small hollow areas and the areas with the diameter smaller than half of the image width into the most similar adjacent areas according to the neighborhood similarity, and re-marking pixels according to the marks of bottom layer superpixels, namely, the pixels in the same bottom layer superpixel are endowed with the labels of the affiliated bottom layer superpixels to obtain a further segmentation result; respectively expanding the grain loading line and the grain surface in the segmentation result, respectively covering the grain loading line and the grain surface by using a rectangular frame with the width of the image, establishing a Gaussian mixture model for the foreground and the background, and respectively refining the grain loading line and the grain surface by using a GrabCT algorithm to obtain a final segmentation result; and finally, performing linear fitting on the lower boundary of the grain loading line in the final segmentation result, and taking the fitted straight line as the lower boundary of the grain loading line.
And finally, calculating the overall height of the grain surface.
FIG. 3 is a schematic diagram of the fitting of the lower boundary of the grain loading line and the calculation of the overall height of the grain surface, as shown in the figure: comprises a grain loading line, a fitted lower boundary straight line and an upper boundary of grain surface.
The specific steps of the fitting of the lower boundary of the grain loading line in the embodiment are as follows: and obtaining the coordinates of the lower boundary pixel points of the grain containing line, fitting the lower boundary of the grain containing line by using a linear least square method to obtain a fitted lower boundary straight line of the grain containing line, and replacing the lower boundary of the grain containing line in the segmentation result with the straight line to remove the outer points and the noise points.
In this embodiment, the overall height of the grain surface is the average distance between the grain loading line and the grain surface, and the calculation steps are as follows: firstly, calculating the vertical distance between each pixel point of the upper boundary of the grain surface and the fitted lower boundary straight line of the grain loading line; and then, calculating the average value of the average values to be used as the average distance between the grain loading line and the grain surface to obtain the overall height of the grain surface.
After the overall height of the grain surface is obtained, comparing the current overall height of the grain surface with the historical overall height of the grain surface, and if the difference between the current overall height of the grain surface and the historical overall height of the grain surface is greater than a set threshold value, giving an alarm to a manager; otherwise, recording the current data. For example: assuming that the average distance from the current grain loading line to the grain surface is DcAverage distance of history is DhIf there is a difference between R and Dc-DhIf R is larger than the specified threshold value, the grain level is considered to have changed greatly, and an alarm is sent to a manager, otherwise, the record is correctThe previous data.
Specific validation experiments are given below:
10 images in a bin are collected, the size of each image is 1080 multiplied by 1920 pixels, and 10 images are shot under different illumination and preset angles. And selecting 6 images as a training set and 4 images as a test set, and manually marking to obtain the Ground Truth.
In order to remove the influence caused by uneven illumination, areas with a width of 0.1 times of the width of each of 20 pixels and two sides of the upper and lower boundaries of the image are cut out, and the image is scaled to 360 × 640 pixels.
The mean shift method is adopted to generate the high-level superpixel of the image, and the related three parameters are respectively set as: hr 13, hs 11, minReg 100; the method comprises the following steps of generating bottom layer superpixels of an image by adopting an SLIC method, wherein the related two parameters are respectively set as: 1000 and 10.
Acquiring CIE LAB color characteristics of the image, and storing the CIE LAB color characteristics and the generated super-pixel mark into a local file; the final characteristics of the bottom layer superpixels are obtained according to the method, and the bottom layer superpixels are used for training an SVM classifier model to obtain a trained SVM classifier.
After image preprocessing, superpixel generation, feature extraction and selection operations are carried out on the images of the test set, inputting the images into a trained SVM classifier for image segmentation to obtain initial segmentation results of grain loading lines and grain surfaces; then, removing hollow areas with the area smaller than 100 pixels, removing areas with the diameter smaller than 0.5 time of the image width, merging the areas into the most similar adjacent areas according to the neighborhood similarity, and re-marking pixels according to the marks of the bottom layer superpixels, namely, the pixels in the same bottom layer superpixel are endowed with the labels of the affiliated bottom layer superpixels to obtain further segmentation results; and finally, thinning the grain loading line and the grain surface by using a GrabCut algorithm, and selecting the weight of the smoothing term as 20 to obtain the final segmentation result.
First, the final segmentation results are tested: and evaluating the final segmentation result by adopting evaluation standards in four image segmentation fields. As shown in table 1, respectively: PRI (proliferative rank index), VOI (variation of information), GCE and BDE; wherein, the PRI is in the interval [0,1], and the larger the value is, the higher the segmentation accuracy is; the VOI is in the range of [0, + ∞ ], and the smaller the value is, the better the segmentation effect is; the smaller the values of BDE and GCE are, the better the segmentation effect is, and the closer the result is to the Ground Truth.
TABLE 1 evaluation Table of segmentation results
As can be seen from Table 1, the segmentation result of the method of the present invention is very close to the result of the artificial marking on four evaluation indexes, so that the effectiveness of the method of the present invention can be demonstrated.
Secondly, carrying out a grain surface descending test: fixing a camera with a holder on a wall, setting corresponding parameters, placing a liftable wood board 15 m away from the camera, uniformly placing wheat 2 cm thick on the wood board, simulating the inner wall of the bin by using a white baffle with red mark lines, and simulating the grain surface to descend by lifting the wood board on which the grains are placed. Through three tests, the grain surface is reduced by 12 cm each time, and the detection results are shown in table 2:
TABLE 2 average distance test results table
In table 2, the heights of the grain surfaces were obtained under the condition that the grain surfaces were respectively lowered by 0.12 m, the average distance is the average distance between the marking line and the grain surfaces, calculated according to the image width of 640 pixels, R is the difference between the current average distance and the average distance of the history, and the threshold value thereof is set to 61.9 (calculated by experiment, 61.9 is approximately equal to the actual height of 0.1 m).
As can be seen from table 2: in the experimental result, the difference R between the second measurement and the first measurement is 70.42, and the difference R between the third measurement and the second measurement is 73.09, which are both greater than the set threshold value 61.9; the difference of every two actual drops of the grain and the flour is 0.12 m, and the difference is larger than the set threshold value of 0.1 m. The experimental results show that: the actual grain surface reduction corresponds to the change of the number of pixels, the requirement of grain quantity change monitoring can be met, and the method is proved to be feasible.
The invention can judge whether the grain storage quantity of the granary changes or not by calculating the average distance between the grain loading line and the grain surface at the grain outlet and comparing the average distance with the preset calculation result of one time or the previous time, and has the advantages of high detection speed and low energy consumption.
The invention can also be combined with a detection method based on infrared laser scanning, when the whole height of the grain surface is found to be reduced, the infrared laser scanner can be used for scanning the whole grain bin to obtain the accurate grain volume, thereby prolonging the service life of the infrared laser.
Example 2
In this embodiment, the difference from embodiment 1 is only that, when calculating the overall height of the grain surface, the area between the grain loading line and the grain surface is adopted, and the calculation steps are as follows: firstly, calculating the vertical distance between each pixel point of the upper boundary of the grain surface and the fitted lower boundary straight line of the grain loading line; and then, calculating the area between the grain loading line and the grain surface by using an integral mode to obtain the overall height of the grain surface.
After the overall height of the grain surface is obtained, comparing the current overall height of the grain surface with the historical overall height of the grain surface, and if the difference between the current overall height of the grain surface and the historical overall height of the grain surface is greater than a set threshold value, giving an alarm to a manager; otherwise, recording the current data. For example: assuming that the area from the current grain loading line to the grain surface is AcArea of history record is AhIf there is a difference between R and Ac-AhIf R is larger than the specified threshold value, the grain level is considered to have changed greatly, and an alarm is sent to a manager, otherwise, the current data is recorded.
Other steps and contents of this embodiment are the same as those of embodiment 1, and thus are not described again.
Specific validation experiments are given below:
10 images in a bin are collected, the size of each image is 1080 multiplied by 1920 pixels, and 10 images are shot under different illumination and preset angles. And selecting 6 images as a training set and 4 images as a test set, and manually marking to obtain the Ground Truth.
In order to remove the influence caused by uneven illumination, areas with a width of 0.1 times of the width of each of 20 pixels and two sides of the upper and lower boundaries of the image are cut out, and the image is scaled to 360 × 640 pixels.
The mean shift method is adopted to generate the high-level superpixel of the image, and the related three parameters are respectively set as: hr 13, hs 11, minReg 100; the method comprises the following steps of generating bottom layer superpixels of an image by adopting an SLIC method, wherein the related two parameters are respectively set as: 1000 and 10.
Acquiring CIE LAB color characteristics of the image, and storing the CIE LAB color characteristics and the generated super-pixel mark into a local file; the final characteristics of the bottom layer superpixels are obtained according to the method, and the bottom layer superpixels are used for training an SVM classifier model to obtain a trained SVM classifier.
After image preprocessing, superpixel generation, feature extraction and selection operations are carried out on the images of the test set, inputting the images into a trained SVM classifier for image segmentation to obtain initial segmentation results of grain loading lines and grain surfaces; then, removing hollow areas with the area smaller than 100 pixels, removing areas with the diameter smaller than 0.5 time of the image width, merging the areas into the most similar adjacent areas according to the neighborhood similarity, and re-marking pixels according to the marks of the bottom layer superpixels, namely, the pixels in the same bottom layer superpixel are endowed with the labels of the affiliated bottom layer superpixels to obtain further segmentation results; and finally, thinning the grain loading line and the grain surface by using a GrabCut algorithm, and selecting the weight of the smoothing term as 20 to obtain the final segmentation result.
First, the final segmentation results are tested: and evaluating the final segmentation result by adopting evaluation standards in four image segmentation fields. As shown in table 3, respectively: PRI (proliferative rank index), VOI (variation of information), GCE and BDE; wherein, the PRI is in the interval [0,1], and the larger the value is, the higher the segmentation accuracy is; the VOI is in the range of [0, + ∞ ], and the smaller the value is, the better the segmentation effect is; the smaller the values of BDE and GCE are, the better the segmentation effect is, and the closer the result is to the Ground Truth.
TABLE 3 evaluation Table of segmentation results
As can be seen from Table 3, the segmentation result of the method of the present invention is very close to the result of the artificial labeling on four evaluation indexes, so that the effectiveness of the method of the present invention can be demonstrated.
Secondly, carrying out a grain surface descending test: fixing a camera with a holder on a wall, setting corresponding parameters, placing a liftable wood board 15 m away from the camera, uniformly placing wheat 2 cm thick on the wood board, simulating the inner wall of the bin by using a white baffle with red mark lines, and simulating the grain surface to descend by lifting the wood board on which the grains are placed. Through three tests, the grain surface is reduced by 12 cm each time, and the detection results are shown in table 4:
table 4 table of area test results
In table 4, the grain level height was obtained when the grain level was lowered by 0.12 m, respectively, the area of the region between the marker line and the grain level, calculated in terms of 640 pixels in image width, R was the difference between the current area and the area of the region of the history, and the threshold value was set to 37640 (37640 was approximately equal to 0.1 m in actual height, estimated by experiment).
As can be seen from table 4: in the experimental results, the difference R between the second measurement and the first measurement is 45071, and the difference R between the third measurement and the second measurement is 46776, which are both greater than the set threshold 37640; the difference of every two actual drops of the grain and the flour is 0.12 m, and the difference is larger than the set threshold value of 0.1 m. The experimental results show that: the actual grain surface reduction corresponds to the change of the number of pixels, the requirement of grain quantity change monitoring can be met, and the method is proved to be feasible.
The invention can judge whether the grain storage quantity of the granary changes or not by calculating the area between the grain loading line and the grain surface at the grain outlet and comparing the area with the preset calculation result of one time or the previous time, and has the advantages of high detection speed and low energy consumption.
The present invention has been described in relation to particular embodiments thereof, but the invention is not limited to the described embodiments. In the thought given by the present invention, the technical means in the above embodiments are changed, replaced, modified in a manner that is easily imaginable to those skilled in the art, and the functions are basically the same as the corresponding technical means in the present invention, and the purpose of the invention is basically the same, so that the technical scheme formed by fine tuning the above embodiments still falls into the protection scope of the present invention.
Claims (7)
1. A computer vision-based automatic monitoring method for grain storage quantity change of a granary comprises the following steps:
(1) collecting grain surface images at a grain outlet; the grain surface image comprises a reference line above the grain surface of the grain outlet, and the reference line is a grain loading line or a marking line;
(2) calculating the overall height of the grain by processing the grain image, and comparing the overall height of the grain with the overall height of the grain calculated at one time or the previous time;
(3) if the whole height of the grain surface is reduced, the number of stored grains in the bin is judged to be reduced;
the process of processing the grain surface image comprises the following steps: extracting the characteristics of all sample images; training an SVM classifier model according to the extracted sample image characteristics; then, segmenting the grain surface image to be processed by utilizing a trained SVM classifier model to obtain a reference line and grain surfaces;
the process of segmenting the grain and flour image to be processed comprises the following steps: preprocessing an image; generating high-level superpixels and bottom-level superpixels of the image; extracting bottom layer super pixel characteristics of the image; the process of extracting the bottom layer super pixel characteristics of the image comprises the following steps: when extracting the characteristics of the bottom layer superpixel, taking the high layer superpixel as space limitation, if the neighborhood pixels of one bottom layer superpixel are all under the same high layer superpixel, the final characteristics of the bottom layer superpixel are the weighted average of the characteristics of the neighborhood superpixel; otherwise, the color mean of the pixels within the underlying superpixel is used as its final feature.
2. The computer vision-based automatic monitoring method for the change of the grain quantity in the granary, according to claim 1, wherein: the integral height of the grain surface refers to the average distance between the reference line and the grain surface, and the average distance is obtained by calculating the average value of the vertical distances from each point of the upper boundary of the grain surface to the lower boundary of the reference line; the overall height reduction of the grain surface means that the average distance between the currently calculated reference line and the grain surface is larger than a preset or previous calculation result.
3. The computer vision-based automatic monitoring method for the change of the grain quantity in the granary, according to claim 1, wherein: the integral height of the grain surface refers to the area between the reference line and the grain surface, and the area is obtained in an integral mode after the vertical distance from each point of the upper boundary of the grain surface to the lower boundary of the reference line is calculated; the integral height reduction of the grain surface means that the area of the area between the currently calculated reference line and the grain surface is larger than the preset calculation result or the previous calculation result.
4. The method of automatically monitoring changes in the quantity of grain stored in a grain bin based on computer vision of claim 2 or 3, wherein the method comprises the following steps: the process of obtaining the reference line and the grain surface further comprises the following steps: denoising, refining and fitting the lower boundary of the reference line.
5. The computer vision-based automatic monitoring method for the change of the grain quantity in the granary, according to claim 4, wherein: the denoising processing process comprises the following steps: and removing small hollow areas and areas with the width less than half of the image width in the segmentation result by using the prior information that the reference line and the grain surface are in a belt shape, and merging the small hollow areas and the areas with the width less than half of the image width into the most similar adjacent areas according to the neighborhood similarity to obtain a further segmentation result.
6. The computer vision-based automatic monitoring method for the change of the grain quantity in the granary, according to claim 5, wherein: the process of the refinement processing comprises the following steps: and respectively expanding the reference line and the grain surface in the further segmentation result, then respectively covering the reference line and the grain surface by using a rectangular frame with the width being the image width, and respectively thinning the reference line and the grain surface by a segmentation method based on graph theory to obtain a final segmentation result.
7. The computer vision-based automatic monitoring method for the change of the grain quantity in the granary according to claim 6, wherein: the process of fitting the lower boundary of the reference line comprises the following steps: and performing linear fitting on the lower boundary of the reference line in the final segmentation result, and taking the fitted straight line as the lower boundary of the reference line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810621000.1A CN109472261B (en) | 2018-06-15 | 2018-06-15 | Computer vision-based automatic monitoring method for grain storage quantity change of granary |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810621000.1A CN109472261B (en) | 2018-06-15 | 2018-06-15 | Computer vision-based automatic monitoring method for grain storage quantity change of granary |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109472261A CN109472261A (en) | 2019-03-15 |
| CN109472261B true CN109472261B (en) | 2020-09-18 |
Family
ID=65660023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810621000.1A Active CN109472261B (en) | 2018-06-15 | 2018-06-15 | Computer vision-based automatic monitoring method for grain storage quantity change of granary |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109472261B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046570B (en) * | 2019-04-12 | 2022-09-30 | 河南工业大学 | Method and device for dynamically supervising grain stock of granary |
| CN110008947B (en) * | 2019-04-12 | 2021-06-29 | 河南工业大学 | Granary grain quantity monitoring method and device based on convolutional neural network |
| CN110084186A (en) * | 2019-04-25 | 2019-08-02 | 中信梧桐港供应链管理有限公司 | A kind of warehouse remote supervisory method and device |
| CN111200723B (en) * | 2020-01-07 | 2021-07-23 | 苏州恒志汇智能科技有限公司 | Progressive die arch material monitoring method, device, equipment and storage medium |
| CN111380461A (en) * | 2020-04-21 | 2020-07-07 | 南京理工大学 | Cable winding state detection system |
| CN111967473B (en) * | 2020-06-29 | 2023-01-20 | 山东浪潮通软信息科技有限公司 | Grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching |
| CN112734826B (en) * | 2020-12-29 | 2022-05-31 | 华信咨询设计研究院有限公司 | Grain quantity estimation method based on deep learning and LSD (least squares-based) linear detection algorithm |
| CN114066967B (en) * | 2022-01-11 | 2022-04-22 | 安徽高哲信息技术有限公司 | Training method and device of volume estimation model and volume estimation method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020014116A1 (en) * | 1999-02-26 | 2002-02-07 | Campbell Ronald H. | Methods and systems for measuring crop density |
| CN1963377A (en) * | 2006-10-30 | 2007-05-16 | 朱阳明 | Measuring method for amount of grain reserve in grain depot |
| CN103968762A (en) * | 2013-02-06 | 2014-08-06 | 普适微芯科技（北京）有限公司 | Granary grain capacity monitoring system based on digital image processing |
| CN104899854B (en) * | 2014-03-05 | 2018-01-16 | 航天信息股份有限公司 | The detection method and device of heap grain altitude line |
| CN105466523B (en) * | 2014-09-12 | 2019-09-17 | 航天信息股份有限公司 | The measurement method and device of heap grain height based on single camera image |
| CN107036687B (en) * | 2017-03-08 | 2020-08-04 | 湖北叶威(集团)智能科技有限公司 | Vision-based grain storage quantity monitoring method and device |
| CN107272899B (en) * | 2017-06-21 | 2020-10-30 | 北京奇艺世纪科技有限公司 | VR (virtual reality) interaction method and device based on dynamic gestures and electronic equipment |
-
2018
- 2018-06-15 CN CN201810621000.1A patent/CN109472261B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109472261A (en) | 2019-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109472261B (en) | Computer vision-based automatic monitoring method for grain storage quantity change of granary | |
| CN106651872B (en) | Pavement crack identification method and system based on Prewitt operator | |
| CN107506798B (en) | Water level monitoring method based on image recognition | |
| CN108759973B (en) | Water level measuring method | |
| CN107633516B (en) | Method and device for identifying road surface deformation diseases | |
| CA3006240C (en) | A stepwise refinement detection method for pavement cracks | |
| CN114419025A (en) | Fiberboard quality evaluation method based on image processing | |
| CN108564085B (en) | Method for automatically reading of pointer type instrument | |
| CN110008947B (en) | Granary grain quantity monitoring method and device based on convolutional neural network | |
| CN112149543B (en) | Building dust recognition system and method based on computer vision | |
| CN113324864A (en) | Pantograph carbon slide plate abrasion detection method based on deep learning target detection | |
| CN107798293A (en) | A kind of crack on road detection means | |
| CN103489187A (en) | Quality test based segmenting method of cell nucleuses in cervical LCT image | |
| CN108665468B (en) | Device and method for extracting tangent tower insulator string | |
| CN103674968A (en) | Method and device for evaluating machine vision original-value detection of exterior corrosion appearance characteristics of material | |
| CN102393902A (en) | Vehicle color detection method based on H_S two-dimensional histogram and regional color matching | |
| CN101739549A (en) | Face detection method and system | |
| CN112561875A (en) | Photovoltaic cell panel coarse grid detection method based on artificial intelligence | |
| CN114882400A (en) | Aggregate detection and classification method based on AI intelligent machine vision technology | |
| CN113362276A (en) | Visual detection method and system for plate | |
| CN113362274B (en) | Rainfall monitoring and calculating method | |
| CN107369163B (en) | Rapid SAR image target detection method based on optimal entropy dual-threshold segmentation | |
| Powell et al. | Automated road distress detection | |
| CN114820597B (en) | Smelting product defect detection method, device and system based on artificial intelligence | |
| CN116189093A (en) | Slag soil vehicle dust on-line monitoring method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |