CN115187588B - Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus - Google Patents
Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus Download PDFInfo
- Publication number
- CN115187588B CN115187588B CN202211088278.XA CN202211088278A CN115187588B CN 115187588 B CN115187588 B CN 115187588B CN 202211088278 A CN202211088278 A CN 202211088278A CN 115187588 B CN115187588 B CN 115187588B
- Authority
- CN
- China
- Prior art keywords
- image
- foreign matter
- color
- foreground
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 73
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 239000000463 material Substances 0.000 claims abstract description 140
- 230000005540 biological transmission Effects 0.000 claims abstract description 51
- 230000002159 abnormal effect Effects 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 20
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 230000000903 blocking effect Effects 0.000 claims abstract description 9
- 238000012216 screening Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 15
- 239000002245 particle Substances 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000005192 partition Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 7
- 239000003086 colorant Substances 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000032258 transport Effects 0.000 description 3
- PCTMTFRHKVHKIS-BMFZQQSSSA-N (1s,3r,4e,6e,8e,10e,12e,14e,16e,18s,19r,20r,21s,25r,27r,30r,31r,33s,35r,37s,38r)-3-[(2r,3s,4s,5s,6r)-4-amino-3,5-dihydroxy-6-methyloxan-2-yl]oxy-19,25,27,30,31,33,35,37-octahydroxy-18,20,21-trimethyl-23-oxo-22,39-dioxabicyclo[33.3.1]nonatriaconta-4,6,8,10 Chemical compound C1C=C2C[C@@H](OS(O)(=O)=O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2.O[C@H]1[C@@H](N)[C@H](O)[C@@H](C)O[C@H]1O[C@H]1/C=C/C=C/C=C/C=C/C=C/C=C/C=C/[C@H](C)[C@@H](O)[C@@H](C)[C@H](C)OC(=O)C[C@H](O)C[C@H](O)CC[C@@H](O)[C@H](O)C[C@H](O)C[C@](O)(C[C@H](O)[C@H]2C(O)=O)O[C@H]2C1 PCTMTFRHKVHKIS-BMFZQQSSSA-N 0.000 description 2
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229910052755 nonmetal Inorganic materials 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of foreign matter detection, and discloses a foreign matter detection method, a foreign matter detection device, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of images of the transmission band area in the video stream to obtain a plurality of frames of material foreground images; local features of the material foreground images of the frames are extracted in a blocking mode, abnormal blocks are screened, and foreign matter candidate areas of the material foreground images of the frames are obtained; and obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track. The method has the advantages of low false detection rate, high foreign matter recall rate and strong robustness of the detection algorithm.
Description
Technical Field
The present invention relates to the field of foreign object detection technologies, and in particular, to a method and an apparatus for detecting a foreign object, a storage medium, and an electronic device.
Background
In the industrial field, in the process of material transportation of a belt conveyor, unconventional objects except materials, such as metal or nonmetal objects of steel plates, iron sheets, steel bars and the like, can appear. The foreign matters are doped in the materials, so that the subsequent production process is influenced, the transportation problems of belt conveyor blockage, belt tearing, feed opening blockage and the like can be caused, and the production safety and the production efficiency are greatly influenced.
The related belt foreign matter detection means mainly comprises a method based on sensor numerical analysis, deep learning and image processing. The method based on the sensor numerical analysis cannot detect foreign matters of similar types to materials, cannot perform foreign matter visualization, and is high in deployment difficulty and high in cost. The foreign matter detection algorithm based on deep learning depends on a large number of real foreign matter images of the belt, and is high in actual acquisition difficulty and long in period. The belt foreign matter detection method based on single-frame image processing can only detect specific foreign matters and is easy to generate false detection.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, one objective of the present invention is to provide a foreign object detection method, which has the advantages of low false detection rate, high foreign object recall rate, and strong robustness of the detection algorithm.
A second object of the present invention is to provide a foreign object detection apparatus.
A third object of the invention is to propose a computer-readable storage medium.
A fourth object of the invention is to propose an electronic device.
In order to achieve the above object, a method for detecting a foreign object according to a first embodiment of the present invention includes: acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of transmission band area images in the video stream to obtain a plurality of frames of material foreground images; extracting local features of the material foreground images of each frame in a blocking mode, and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of each frame; and obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of the material foreground image of each frame, and determining whether the foreign matter is detected according to the foreign matter transportation track.
According to the foreign matter detection method disclosed by the embodiment of the invention, the foreign matter detection is carried out based on the material foreground image, and compared with the existing detection method based on the whole image, the background interference is effectively reduced, and the false detection rate is reduced; the foreign matter characteristics of each image block are analyzed through the difference characteristics, so that the foreign matters can be identified according to the difference information such as colors, textures, edges and shapes between the foreign matters and materials, and the foreign matter detection recall rate is effectively improved; and on the basis of single-frame foreign matter detection, a foreign matter time sequence trajectory analysis method is introduced, so that the false detection of foreign matters can be effectively reduced, and the algorithm robustness is improved.
In addition, the foreign object detection method proposed according to the above embodiment of the present invention may have the following additional technical features:
according to an embodiment of the present invention, preprocessing a plurality of frames of images of a transmission band area in a video stream to obtain a plurality of frames of foreground images of a material, includes: acquiring a plurality of frames of the transmission band area images in the video stream at preset time intervals; marking the transmission band area in the transmission band area image of each frame to obtain a target area image; aiming at a current frame target area image, carrying out gray difference processing on the current frame target area image and a next frame target area image, and carrying out binarization by using a preset threshold value to obtain a motion difference image; filling the motion area in the motion difference image by using morphological closed operation, and calculating the connected domain area of the filled motion area to obtain a material mask; and overlapping the current frame target area image and the material mask with operation to obtain a current frame material foreground image.
According to one embodiment of the invention, the local features of the foreground image of the current frame material of each frame are extracted in a blocking manner, and abnormal block screening is performed, wherein the abnormal block screening comprises the following steps: dividing the current frame material foreground image by using pre-divided blocks with preset sizes to obtain M multiplied by N image blocks with the preset sizes; calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space; and determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
According to one embodiment of the invention, the size of the pre-portioned blocks is determined based on the material granularityWhereinindicates the width of the pre-partition,indicating the high of the pre-partition,particle size of larger than materialWherein, in the process,a length representing the estimated particle size of the material,a width representing an estimated material particle size;
wherein,m =1,2, 3., M denotes the image block horizontal index, N =1,2, 3., N denotes the image block vertical index, x, y denote the horizontal and vertical coordinates of the vertex at the upper left corner of the image block, respectively, and w, h denote the width and height of the image block, respectively.
According to one embodiment of the present invention, the expression of the color distance characteristic is:
wherein,the color distance characteristic is represented by a color distance,is a function of the inverse cosine of the,representing the image blockThe color mean vector in the RGB color space,representing the color mean vector of the material foreground image in RGB color space, wherein the value range of the color distance characteristic is;
The expression of the color variance characteristic is as follows:
wherein,a color variance characteristic is represented by a color variance of the color image,for the image blockThe color variance of the chrominance component H component in the HSV color space,and the integral color variance of the chromaticity component H component of the material foreground image in the HSV color space.
According to an embodiment of the present invention, the determining the abnormal block in the foreground map of the current frame material according to the color distance characteristic and the color variance characteristic of each image block includes:
judging each image block in the current frame material foreground imageSaid color distance characteristic ofWhether greater than a color distance thresholdThe color variance characteristicWhether greater than a color variance threshold;
When in use≥And is and≥determining the image blockIs the exception block and is notedWherein K =1,2, 3., K denotes an index of the foreign object candidate region, and t denotes a foreground map of the current frame material.
According to one embodiment of the invention, the foreign matter transportation track is obtained based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and the method comprises the following steps: obtaining a foreign matter candidate area in the current frame material foreground imagePosition information of each of the abnormal blocks; determining a preset position according to the transportation direction and the position information, and judging whether the preset position of the next frame of material foreground image is a foreign matter candidate areaWherein, in the process,representing a foreign matter candidate area of the next frame material foreground image, and t +1 representing the next frame material foreground image; if so, recording the track of the foreign matter candidate area, and continuing to perform track association of the foreign matter candidate area of the subsequent frame; if not, the foreign matter candidate area is cleared, and the track association of the foreign matter candidate area is not carried out.
In order to achieve the above object, a second aspect of the present invention provides a foreign object detection apparatus, including: the acquisition module is used for acquiring a video stream of a transmission belt area, and preprocessing a plurality of frames of images of the transmission belt area in the video stream to obtain a plurality of frames of material foreground images; the screening module is used for extracting local characteristics of the material foreground images of the frames in a blocking mode and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of the frames; and the detection module is used for obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track.
To achieve the above object, a third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the foreign object detection method according to the first aspect of the present invention.
In order to achieve the above object, a fourth aspect of the present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the computer program is executed by the processor to implement the foreign object detection method according to the first aspect of the present invention.
Drawings
FIG. 1 is a flow chart of a foreign object detection method according to an embodiment of the present invention;
FIG. 2 is a flow diagram of pre-processing a transport band region image according to one embodiment of the invention;
FIG. 3 is a schematic diagram of obtaining a map of a target area in accordance with one embodiment of the present invention;
FIG. 4 is a schematic flow chart of obtaining a material foreground map according to an embodiment of the present invention;
FIG. 5 is a flow chart of a method of detecting foreign objects in accordance with one embodiment of the present invention;
FIG. 6 is a schematic diagram of a candidate area for a foreign object in a foreground view of a material according to an embodiment of the present invention;
fig. 7 is a schematic view of a foreign object detection apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative and intended to explain the present invention and should not be construed as limiting the present invention.
The foreign object detection method, apparatus, storage medium, and electronic device according to the embodiments of the present invention will be described in detail with reference to fig. 1 to 7 and specific embodiments of the present invention.
Fig. 1 is a flowchart of a foreign object detection method according to an embodiment of the present invention. As shown in fig. 1, the foreign object detection method may include:
s1, acquiring a video stream of a transmission band area, and preprocessing multiple frames of images of the transmission band area in the video stream to obtain multiple frames of material foreground images;
according to the embodiment of the invention, when the foreign matter in the material on the transmission belt is detected, the video stream of the transmission belt area is obtained, the multi-frame transmission belt area image is obtained from the video stream, and the foreign matter detection is carried out on the single-frame transmission belt area image. In order to reduce background interference, the single-frame transmission band area image is preprocessed to obtain a single-frame material foreground image only containing a material area.
For better effect, the camera equipment for collecting the video stream in the area of the transmission belt is installed over the transmission belt, so that the camera equipment can clearly shoot the transportation condition of materials on the transmission belt, the quality of the obtained images in the area of the transmission belt is improved, and the influence of the image quality on the detection of foreign matters is reduced.
In an embodiment of the present invention, as shown in fig. 2, preprocessing a plurality of frames of images of a transmission band region in a video stream to obtain a plurality of frames of foreground images of a material may include:
and S11, acquiring multi-frame transmission band area images in the video stream at preset time intervals.
In order to avoid missing partial materials on a transmission belt when a multi-frame transmission belt area image is acquired from a video stream at a preset time interval i, the preset time interval i can be set according to the running speed of the transmission belt. And acquiring multi-frame transmission band area images from the video stream according to the set time interval i, preprocessing the single-frame transmission band area images, and detecting foreign matters in the preprocessed images.
In the embodiment of the present invention, since the running speed of the transmission belt is relatively uniform and does not change suddenly, i can take 10s, that is, every 10s, one transmission belt region image is obtained from the video stream.
And S12, labeling the transmission band region in the transmission band region image of each frame to obtain a target region image.
Specifically, as shown in FIG. 3, the four vertices of the transmission band region in the single frame transmission band region image are marked clockwise, denoted A, B, C, and D in that order. The quadrilateral ABCD contains a transmission band region. And intercepting the external matrix area marked with the quadrangle ABCD from the image of the transmission belt area, and recording the external matrix area as a matrix A 'B' C 'D'. Pixels other than the transfer band in the matrix a 'B' C 'D' are 0-filled, and a partial image, i.e., a target area map, with respect to only the transfer band is acquired.
And S13, carrying out gray difference processing on the current frame target area image and the next frame target area image, and carrying out binarization by using a preset threshold value to obtain a motion difference image.
Specifically, for the current frame target area map, i.e. the tth frame target area mapAnd its next frame target area map, i.e. t +1 th frame target area mapPerforming gray level processing on the target area image of the transmission belt area image acquired at the t + i second, calculating the gray level value of each pixel of the current frame target area image and the next frame target area image, and subtracting the gray level value of the corresponding pixel point to obtain the gray level difference value of each pixel point, comparing the gray level difference value with a preset threshold value, wherein the gray level difference value is within the range of the preset threshold value, and the gray level value of the difference image at the pixel point is regarded as 0; if the gray difference is outside the preset threshold range, the gray value of the difference image at the pixel point is 255, and a motion difference image is obtained, as shown in fig. 4.
In the embodiment of the present invention, when i =10, the difference processing is performed every 10 s. Wherein, i is 10, and the algorithm speed can be increased.
And S14, filling the motion area in the motion difference image by using morphological closed operation, and calculating the connected domain area of the filled motion area to obtain the material mask.
Specifically, as shown in fig. 4, in the morphological processing algorithm, the operation of performing the dilation process first and then performing the erosion process may be referred to as a closed operation, and the closed operation may be used for filling fine holes in the object, connecting neighboring objects, and smoothing the boundary thereof. And performing morphological closed operation processing on the motion difference image after the binarization processing to fill a motion area in the motion difference image, and connecting images formed by high-numerical-value pixel points with relatively close positions together to obtain a plurality of connected domains. And respectively counting the number of pixel points contained in each connected domain in the motion difference image after filling processing, wherein the connected domain with the largest number of pixel points is a material mask.
In the embodiment of the present invention, the number of times of the expansion processing and the etching processing in the closed-loop operation is not limited, and the optimal processing effect is achieved.
And S15, overlapping the current frame target area image and the material mask with operation to obtain a current frame material foreground image.
Will be whenTarget area map of previous frame, i.e. target area map of t-th frameAnd overlapping with the obtained material mask to obtain a current frame material foreground imageSee fig. 4. And carrying out the processing on the images of the transmission belt areas of each frame to obtain a material foreground image of the images of the transmission belt areas of each frame, and carrying out foreign matter detection on the material foreground image of the images of the transmission belt areas of each frame.
And S2, extracting local features of the material foreground images of the frames in a blocking manner, and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of the frames.
Specifically, local features in the material foreground image are extracted in a blocking mode by means of preset blocks, abnormal block screening is conducted on the local features of each image block, and abnormal blocks of the material foreground image are detected, wherein each abnormal block in the material foreground image is a foreign matter candidate area of the material foreground image.
In an embodiment of the present invention, as shown in fig. 5, extracting local features of a foreground image of a current frame material in each frame in a block manner, and performing abnormal block screening may include:
segmenting a current frame material foreground image by utilizing pre-segmentation blocks with preset sizes to obtain M multiplied by N image blocks with preset sizes;
calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space;
and determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
Referring to fig. 6, a foreground image of a current frame material is segmented by using pre-partitioned blocks of a preset size to obtain M × N image blocks of the preset size, where the size of the pre-partitioned blocks may be determined according to the material granularity. Because in the industrial production process, the conveying belt generally transports the same materials, the colors of the materials are generally close, and the main difference is the material granularity and the light color. The foreign matters usually include misplaced equipment, aged and fallen mechanical parts, belt fences and other objects, and have larger differences from the colors and forms of the materials. Therefore, the color distance characteristic in the RGB color space and the color variance characteristic in the HSV color space of each image block can be calculated. And detecting abnormal blocks in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block.
In an embodiment of the invention, the size of the pre-partitions is determined based on the material granularityWherein, in the process,indicates the width of the pre-partition,indicating the high of the pre-partition,greater than estimated material particle sizeWhereinindicates the length of the estimated particle size of the material,a width representing an estimated material particle size;
wherein,m =1,2, 3., M denotes an image block horizontal index, N =1,2, 3., N denotes an image block vertical index, x, y denote the horizontal and vertical coordinates of the vertex at the upper left corner of the image block, respectively, and w, h denote the width and height of the image block, respectively.
It should be noted that m and n are positive integers, and the value range is determined according to the horizontal number and the vertical number of the image blocks in the material foreground image.
In an embodiment of the invention, the pre-tiles are squares, wherein,,representing the side length of a preset pre-partition block.
In an embodiment of the present invention, the expression of the color distance characteristic is:
wherein,a color distance characteristic is represented by a color distance characteristic,is a function of the inverse cosine of the,representing image blocksA color mean vector in the RGB color space,representing colors of material foreground images in RGB color spaceMean vector, color distance characteristics of value range of;
The expression for the color variance feature is:
wherein,the color variance characteristic is represented by a color variance characteristic,is an image blockThe color variance of the chrominance component H component in the HSV color space,and the integral color variance of the chromaticity component H component of the material foreground image in the HSV color space.
Specifically, the color distance characteristic of each image block in the RGB color space is calculated using the above expression of the color distance characteristic and the expression of the color variance characteristicAnd color variance characteristics in HSV color space. According to the color distance thresholdAnd color variance thresholdColor distance for each image blockCharacteristic ofAnd color variance characteristicsAnd (5) judging to determine an abnormal block in the current frame material foreground image.
In an embodiment of the present invention, as shown in fig. 5, determining an abnormal block in the foreground map of the current frame material according to the color distance characteristic and the color variance characteristic of each image block may include:
judging each image block in the foreground image of the current frame materialColor distance characteristic ofWhether greater than a color distance thresholdColor variance characteristicsWhether greater than a color variance threshold;
When the temperature is higher than the set temperature≥And is and≥then, image blocks are determinedIs an abnormal block and is recorded asWherein K =1,2, 3., K denotes an index of the foreign object candidate region, and t denotes a foreground map of the current frame material.
Specifically, each image blockColor distance characteristic ofDistance from color thresholdComparing, color variance characterizationAnd color variance thresholdA comparison is made. Judging image blocksColor distance characteristic ofWhether greater than a color distance thresholdColor variance characteristicsWhether greater than a color variance threshold. Wherein the color distance characteristicGreater than a color distance thresholdAnd color variance characteristicsIs also greater than the color variance thresholdImage block ofIs an exception block.
In addition, according to the image blocksM and n of (c), the row and column where the image block that is the abnormal block is located, i.e., the location of the abnormal block, can be determined.
And S3, obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track.
In an embodiment of the present invention, as shown in fig. 5, obtaining a foreign object transportation track based on a transportation direction of a conveyor and a foreign object candidate region of a foreground map of each frame of material may include:
obtaining a foreign matter candidate area in a current frame material foreground imagePosition information of each abnormal block;
determining a preset position according to the transportation direction and the position information, and judging whether the preset position of the next frame of material foreground image is a foreign matter candidate areaWhereinrepresenting a foreign matter candidate area of the next frame material foreground image, and t +1 representing the next frame material foreground image;
if so, recording the foreign matter candidate area track, and continuing to perform track association of the foreign matter candidate area of the subsequent frame;
if not, the foreign matter candidate area is removed, and the track association of the foreign matter candidate area is not carried out.
Specifically, a foreign matter candidate area in a current frame material foreground image is obtainedPosition information of each abnormal blockDetermining the preset position of the foreign object block corresponding to the foreground image of the next frame of material according to the conveying direction of the conveying belt, the conveying speed and the preset time interval as well as the foreground image of the current frame of material, namely the position information of the foreign object block in the foreground image of the tth frame of materialIn order to prevent the occurrence of other problems such as jamming of the conveyor belt, the predetermined position may further includeAnd/orTo prevent missed detection. After the preset position is determined, judging whether the preset position of the next frame of material foreground image is a foreign matter candidate area or not. If the preset position in the next frame of material foreground image is a foreign matter candidate areaRecording the foreign object candidate area track and continuingAnd performing track association of foreign matter candidate areas of the subsequent frames. If the preset position of the next frame of material foreground image is not the foreign matter candidate areaAnd removing the foreign matter candidate area, and not performing the track correlation of the foreign matter candidate area so as to improve the accuracy of foreign matter detection and reduce the false detection rate.
Wherein the candidate foreign body track obtained by correlation is recorded asWherein id represents the index corresponding to the track.The following conditions need to be satisfied: the starting position of the track is the starting position of material transportation in the image; the track end position is a material transportation end position in the image.
Compared with the existing detection method based on the whole image, the foreign matter detection method based on the material foreground image effectively reduces background interference and lowers false detection rate; foreign matter characteristics of each image block are analyzed through difference characteristics, so that the foreign matters can be identified according to difference information such as colors, textures, edges and shapes between the foreign matters and materials, and the foreign matter detection recall rate is effectively improved; and on the basis of single-frame foreign matter detection, a foreign matter time sequence trajectory analysis method is introduced, so that the false detection of foreign matters can be effectively reduced, and the algorithm robustness is improved.
The invention also provides a foreign matter detection device.
Fig. 7 is a schematic view of a foreign object detection apparatus according to an embodiment of the present invention. As shown in fig. 7, the foreign object detection apparatus 100 may include an acquisition module 10, a screening module 20, and a detection module 30.
The acquiring module 10 is configured to acquire a video stream of a transmission band region, and pre-process a plurality of frames of images of the transmission band region in the video stream to obtain a plurality of frames of material foreground images; the screening module 20 is configured to extract local features of each frame of material foreground image in a block manner, and perform abnormal block screening to obtain a foreign matter candidate region of each frame of material foreground image; the detection module 30 is configured to obtain a foreign object transportation track based on the transportation direction of the conveyor belt and the foreign object candidate region of each frame of the material foreground map, and determine whether to detect a foreign object according to the foreign object transportation track.
It should be noted that, for other specific embodiments of the foreign object detection apparatus according to the embodiment of the present invention, reference may be made to specific embodiments of the foreign object detection method according to the above-described embodiment of the present invention.
Compared with the existing detection method based on the whole image, the foreign matter detection device provided by the embodiment of the invention effectively reduces background interference and lowers false detection rate; foreign matter characteristics of each image block are analyzed through difference characteristics, so that the foreign matters can be identified according to difference information such as colors, textures, edges and shapes between the foreign matters and materials, and the foreign matter detection recall rate is effectively improved; and on the basis of single-frame foreign matter detection, a foreign matter time sequence trajectory analysis method is introduced, so that the false detection of foreign matters can be effectively reduced, and the algorithm robustness is improved.
The invention also provides a computer readable storage medium.
In this embodiment, a computer-readable storage medium has stored thereon a computer program, which corresponds to the above-mentioned foreign object detection method, and which, when executed by a processor, implements the foreign object detection method as described above.
The invention also provides the electronic equipment.
In this embodiment, the electronic device comprises a processor, a memory and a computer program stored on the memory, which when executed by the processor implements the foreign object detection method as described above.
The storage medium and the electronic device of the embodiment of the invention utilize the foreign matter detection method to realize the foreign matter detection of the transmission belt.
It should be noted that the logic and/or steps shown in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and are therefore not to be considered limiting of the invention.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (8)
1. A foreign object detection method, characterized in that the method comprises:
acquiring a video stream of a transmission band area, and preprocessing a plurality of frames of images of the transmission band area in the video stream to obtain a plurality of frames of material foreground images;
extracting local features of the material foreground images of each frame in a blocking manner, and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of each frame;
obtaining a foreign matter transportation track based on the transportation direction of a conveyor belt and the foreign matter candidate area of each frame of the material foreground image, and determining whether foreign matters are detected according to the foreign matter transportation track;
extracting local features of the material foreground images of each frame in a blocking mode, and screening abnormal blocks, wherein the method comprises the following steps:
segmenting a current frame material foreground image by utilizing pre-segmentation blocks with preset sizes to obtain M multiplied by N image blocks with the preset sizes;
calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space;
determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block;
the expression of the color distance feature is:
wherein,a feature representing a distance of said color is provided,is a function of the inverse cosine of the,representing image blocks in the m-th row and n-th columnA color mean vector in an RGB color space, wherein M =1,2, 3.., M, M denotes the tile lateral index, N =1,2, 3.., N, N denotes the tile longitudinal index,representing the color mean vector of the material foreground image in an RGB color space, wherein the value range of the color distance characteristic is;
The expression of the color variance characteristic is as follows:
wherein,a feature representing the variance of the color is presented,for the image blockThe color variance of the chrominance component H component in the HSV color space,and the integral color variance of the chromaticity component H component of the material foreground image in the HSV color space.
2. The foreign object detection method of claim 1, wherein preprocessing the images of the plurality of frames of the transmission band region in the video stream to obtain a plurality of frames of the foreground image of the material comprises:
acquiring multiple frames of the transmission band area images in the video stream at preset time intervals;
marking the transmission band area in the transmission band area image of each frame to obtain a target area image;
aiming at a current frame target area image, carrying out gray level difference processing on the current frame target area image and a next frame target area image, and carrying out binarization by using a preset threshold value to obtain a motion difference image;
filling the motion area in the motion difference image by using morphological closed operation, and calculating the connected domain area of the filled motion area to obtain a material mask;
and overlapping the current frame target area image and the material mask with operation to obtain a current frame material foreground image.
3. The foreign object detection method according to claim 2, wherein the size of the pre-partition is determined based on the particle size of the materialWhereinindicates the width of the pre-partition,indicating the high of the pre-partition,particle size of larger than materialWhereinindicates the length of the estimated particle size of the material,a width representing an estimated material particle size;
wherein,m =1,2,3,. Multidot., M represents the image block horizontal index, N =1,2,3,. Multidot., N represents the image block longitudinal index, x, y represent the horizontal and vertical coordinates of the vertex at the upper left corner of the image block, respectively, and w, h represent the width and height of the image block, respectively.
4. The method according to claim 3, wherein the determining the abnormal block in the foreground map of the current frame material according to the color distance feature and the color variance feature of each image block comprises:
judging each image block in the current frame material foreground imageSaid color distance characteristic ofWhether or not it is greater than or equal to a color distance thresholdThe color variance characteristicWhether or not it is greater than or equal to the color variance threshold;
5. The foreign matter detection method according to claim 4, wherein obtaining the foreign matter transport trajectory based on the transport direction of the transport belt and the foreign matter candidate region of the material foreground map of each frame comprises:
obtaining a foreign matter candidate area in the foreground image of the current frame materialPosition information of each of the abnormal blocks;
determining a preset position according to the transportation direction and the position information, and judging whether the preset position of the next frame of material foreground image is a foreign matter candidate areaWhereinrepresenting a foreign matter candidate area of the next frame material foreground image, and t +1 representing the next frame material foreground image;
if so, recording the track of the foreign matter candidate area, and continuing to perform track association of the foreign matter candidate area of the subsequent frame;
if not, the foreign matter candidate area is cleared, and the track association of the foreign matter candidate area is not carried out.
6. A foreign object detection apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring a video stream of a transmission belt area, and preprocessing a plurality of frames of images of the transmission belt area in the video stream to obtain a plurality of frames of material foreground images;
the screening module is used for extracting local characteristics of the material foreground images of each frame in a blocking mode and screening abnormal blocks to obtain foreign matter candidate areas of the material foreground images of each frame;
the detection module is used for obtaining a foreign matter transportation track based on the transportation direction of the conveyor belt and the foreign matter candidate area of each frame of material foreground image, and determining whether the foreign matter is detected according to the foreign matter transportation track;
the screening module is specifically used for segmenting a current frame material foreground image by utilizing pre-segmentation blocks with preset sizes to obtain M multiplied by N image blocks with the preset sizes;
calculating the color distance characteristic of each image block in an RGB color space and the color variance characteristic of each image block in an HSV color space;
determining an abnormal block in the foreground image of the current frame material according to the color distance characteristic and the color variance characteristic of each image block;
the expression of the color distance characteristic is as follows:
wherein,the color distance characteristic is represented by a color distance,in the form of an inverse cosine function,image block for representing m row and n columnA color mean vector in an RGB color space, wherein M =1,2, 3.., M, M denotes the tile lateral index, N =1,2, 3.., N, N denotes the tile longitudinal index,representing the color mean vector of the material foreground image in an RGB color space, wherein the value range of the color distance characteristic is;
The expression of the color variance characteristic is as follows:
wherein,a color variance characteristic is represented by a color variance of the color image,for the image blockThe color variance of the chrominance component H component in the HSV color space,and the integral color variance of the chromaticity component H component of the material foreground image in the HSV color space.
7. A computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements a foreign object detection method according to any one of claims 1 to 5.
8. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the computer program, when executed by the processor, implements the foreign object detection method of any of claims 1-5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211088278.XA CN115187588B (en) | 2022-09-07 | 2022-09-07 | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211088278.XA CN115187588B (en) | 2022-09-07 | 2022-09-07 | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115187588A CN115187588A (en) | 2022-10-14 |
CN115187588B true CN115187588B (en) | 2022-12-09 |
Family
ID=83523803
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211088278.XA Active CN115187588B (en) | 2022-09-07 | 2022-09-07 | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115187588B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115947066B (en) * | 2023-03-14 | 2023-06-20 | 合肥金星智控科技股份有限公司 | Belt tearing detection method, device and system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104700430A (en) * | 2014-10-05 | 2015-06-10 | 安徽工程大学 | Method for detecting movement of airborne displays |
CN106778595B (en) * | 2016-12-12 | 2020-04-07 | 河北工业大学 | Method for detecting abnormal behaviors in crowd based on Gaussian mixture model |
CN110555868A (en) * | 2019-05-31 | 2019-12-10 | 南京航空航天大学 | method for detecting small moving target under complex ground background |
CN113486714B (en) * | 2021-06-03 | 2022-09-02 | 荣耀终端有限公司 | Image processing method and electronic equipment |
-
2022
- 2022-09-07 CN CN202211088278.XA patent/CN115187588B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN115187588A (en) | 2022-10-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114937055B (en) | Image self-adaptive segmentation method and system based on artificial intelligence | |
US9946941B2 (en) | Lane detection | |
TWI409718B (en) | Method of locating license plate of moving vehicle | |
CN112184744B (en) | Display screen edge defect detection method and device | |
CN109146860B (en) | Full-automatic mechanical equipment installation leakage detection method and device | |
US20070110309A1 (en) | Shadow detection in images | |
CN113221861B (en) | Multi-lane line detection method, device and detection equipment | |
JP4156084B2 (en) | Moving object tracking device | |
CN115187588B (en) | Foreign matter detection method, foreign matter detection device, storage medium, and electronic apparatus | |
JPH0957201A (en) | Specific color region extracting system and specific color region removing system | |
US8000535B2 (en) | Methods and systems for refining text segmentation results | |
JP2022551366A (en) | Method, computer program product and computer readable medium for generating masks for camera streams | |
CN107610132B (en) | Method for removing stains from ancient book document image | |
CN112508913A (en) | Cable section edge detection method based on image detection | |
US10115028B2 (en) | Method and device for classifying an object in an image | |
CN114037730A (en) | Traffic flow statistical method, device, equipment and storage medium | |
Vargas et al. | A shadow removal algorithm for vehicle detection based on reflectance ratio and edge density | |
US20140285662A1 (en) | Image processing apparatus, and method | |
US20150117769A1 (en) | Blob-Encoding | |
CN113516608A (en) | Tire defect detection method and device, and tire detection device | |
Manipriya et al. | Performance analysis of spatial color information for object detection using background subtraction | |
CN114581447B (en) | Conveying belt deviation identification method and device based on machine vision | |
JP2021052238A (en) | Deposit detection device and deposit detection method | |
CN115661194A (en) | Moving object extraction method, system, electronic device and medium | |
CN110223299B (en) | Abrasive particle segmentation method based on deposition process |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |