CN115330687A - Workpiece counting method and device based on FIDT and computer readable storage medium - Google Patents

Workpiece counting method and device based on FIDT and computer readable storage medium Download PDF

Info

Publication number
CN115330687A
CN115330687A CN202210830288.XA CN202210830288A CN115330687A CN 115330687 A CN115330687 A CN 115330687A CN 202210830288 A CN202210830288 A CN 202210830288A CN 115330687 A CN115330687 A CN 115330687A
Authority
CN
China
Prior art keywords
target
image
workpiece
fidt
target image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210830288.XA
Other languages
Chinese (zh)
Inventor
翟懿奎
梁雅淇
廖锦锐
江子义
周文略
蒋润锦
张俊亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuyi University
Original Assignee
Wuyi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuyi University filed Critical Wuyi University
Priority to CN202210830288.XA priority Critical patent/CN115330687A/en
Publication of CN115330687A publication Critical patent/CN115330687A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30164Workpiece; Machine component
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30242Counting objects in image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses a workpiece counting method, a workpiece counting device and a computer readable storage medium based on FIDT, wherein the method comprises the following steps: acquiring a target image comprising a plurality of workpiece images, and establishing an FIDT (Fidt specification) according to the target image; acquiring a plurality of target local maximum values from the FIDT according to a preset local maximum value detection strategy, and determining target coordinate information corresponding to each target local maximum value, wherein each target coordinate information represents coordinate information of the central position of each workpiece image; calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm; generating each bounding box according to each instance size; and carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result. According to the embodiment of the application, the workpiece counting is realized by using the FIDT algorithm, and the workpieces which are different in placing posture and dense in placing can be accurately counted, so that the accuracy of the workpiece counting result is improved.

Description

Workpiece counting method and device based on FIDT and computer readable storage medium
Technical Field
The present application relates to, but not limited to, the field of image processing technologies, and in particular, to a method, an apparatus, and a computer-readable storage medium for workpiece counting based on FIDT.
Background
In the process of counting the number of workpieces in a production workshop, a manual counting mode is mostly adopted, the counting mode is complex in task, low in working efficiency and high in labor cost, and counting errors are easily caused by human fatigue and other factors. With the continuous development of artificial intelligence algorithms, a workpiece counting algorithm based on a visual algorithm appears, the automatic counting of workpieces is realized, the consumption of labor cost is reduced, but the traditional visual algorithm is difficult to identify and position the workpieces which are in different postures and are densely arranged, and the accuracy of workpiece counting results is low.
Disclosure of Invention
The embodiment of the application provides a workpiece counting method, a workpiece counting device and a computer readable storage medium based on FIDT, which can effectively improve the accuracy of workpiece counting results.
In a first aspect, an embodiment of the present application provides a workpiece counting method based on an inverse focal length map FIDT, including:
acquiring a target image, and establishing an FIDT (Fidt image data set) according to the target image, wherein the target image comprises a plurality of workpiece images;
acquiring a preset local maximum value detection strategy, acquiring a plurality of target local maximum values from the FIDT according to the local maximum value detection strategy, and determining target coordinate information corresponding to each target local maximum value, wherein each target coordinate information represents coordinate information of the center position of each workpiece image;
calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm;
generating each bounding box according to each instance size;
and carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result.
In some embodiments, the acquiring the target image comprises:
acquiring an image to be processed, and determining an interested area of the image to be processed, wherein the interested area is an area in which a plurality of workpiece images exist in the image to be processed;
and cutting the workpiece image to be processed to obtain the target image, wherein the target image belongs to the region of interest.
In some embodiments, prior to said establishing FIDT from said target image, said method further comprises:
and carrying out image preprocessing on the target image according to a preset preprocessing rule.
In some embodiments, the target image further comprises annotation points corresponding to respective workpiece images; the establishing of the FIDT according to the target image comprises the following steps:
acquiring pixel points of the target image, and calculating Euclidean distances between the pixel points and the labeling points;
and establishing the FIDT according to the Euclidean distances.
In some embodiments, the performing, according to a preset iterative object detection algorithm, target detection processing on each of the enclosure frame and the target image to obtain a target workpiece counting result includes:
acquiring a first image, wherein the first image is an image comprising each surrounding frame, and the first image and the target image have the same size;
attaching a mask to all the bounding boxes in the first image to obtain a second image;
inputting the second image and the target image to a preset object detector for iterative detection processing to obtain a first workpiece counting result;
and when a preset condition is met, determining the sum of all the first workpiece counting results as the target workpiece counting result.
In some embodiments, the preset condition includes at least one of:
the iteration times of the object detector for carrying out iteration detection processing on the second image and the target image exceed a preset iteration time threshold;
alternatively, the first and second electrodes may be,
the first workpiece count result is 0.
In some embodiments, the clustering algorithm is a nearest neighbor rule classification, KNN, algorithm.
In a second aspect, an embodiment of the present application provides an FIDT-based workpiece counting apparatus, including:
the FIDT establishing module is used for acquiring a target image and establishing the FIDT according to the target image, wherein the target image comprises a plurality of workpiece images;
the target coordinate information acquisition module is used for acquiring a preset local maximum detection strategy, acquiring a plurality of target local maxima from the FIDT according to the local maximum detection strategy and determining target coordinate information corresponding to each target local maximum, wherein each piece of target coordinate information represents coordinate information of the center position of each workpiece image;
the example size calculation module is used for calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm;
a bounding box generation module, configured to generate each bounding box according to each instance size;
and the workpiece counting result acquisition module is used for carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result.
In a third aspect, an embodiment of the present application provides an FIDT-based workpiece counting apparatus, including: memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the computer program implementing the FIDT based workpiece counting method according to the first aspect.
In a fourth aspect, embodiments of the present application further provide a computer-readable storage medium storing computer-executable instructions for performing the FIDT-based workpiece counting method according to the first aspect.
The embodiment of the application provides a workpiece counting method, a workpiece counting device and a computer-readable storage medium based on FIDT, wherein the method comprises the following steps: acquiring a target image, and establishing an FIDT (Fidt image data set) according to the target image, wherein the target image comprises a plurality of workpiece images; acquiring a preset local maximum detection strategy, acquiring a plurality of target local maxima from the FIDT according to the local maximum detection strategy, and determining target coordinate information corresponding to each target local maximum, wherein each target coordinate information represents coordinate information of the center position of each workpiece image; calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm; generating each bounding box according to each instance size; and carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result. According to the scheme provided by the embodiment of the application, the workpiece counting can be realized by utilizing the FIDT algorithm, and the accurate counting can be realized for the workpieces which are different in placing posture and dense in placing, so that the accuracy of the workpiece counting result is improved.
Drawings
FIG. 1 is a flowchart illustrating the steps of a FIDT-based workpiece counting method according to one embodiment of the present application;
FIG. 2 is a flowchart illustrating steps provided in another embodiment of the present application for acquiring an image of a target;
FIG. 3 is a flowchart illustrating steps for image pre-processing a target image according to another embodiment of the present application;
FIG. 4 is a flowchart illustrating steps provided in another embodiment of the present application for establishing an FIDT;
FIG. 5 is a flowchart illustrating steps provided in another embodiment of the present application to obtain a target workpiece count result;
FIG. 6 is a block diagram of an FIDT-based workpiece counting apparatus according to another embodiment of the present application;
fig. 7 is a block diagram of a FIDT-based workpiece counting apparatus according to another embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be appreciated that, although functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, the steps shown or described may be performed in a different order than the block divisions in the apparatus, or in the flowcharts. The terms "first," "second," and the like in the description, in the claims, or in the drawings described above, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The application provides a workpiece counting method, a workpiece counting device and a computer readable storage medium based on FIDT, wherein the method comprises the following steps: acquiring a target image, and establishing an FIDT (Fidt image data set) according to the target image, wherein the target image comprises a plurality of workpiece images; acquiring a preset local maximum detection strategy, acquiring a plurality of target local maxima from the FIDT according to the local maximum detection strategy, and determining target coordinate information corresponding to each target local maximum, wherein each target coordinate information represents coordinate information of the center position of each workpiece image; calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm; generating respective bounding boxes according to respective said instance sizes; and carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result. According to the scheme provided by the embodiment of the application, the FIDT algorithm can be utilized to realize workpiece counting, and the workpieces which are placed in different postures and densely can be accurately counted, so that the accuracy of the workpiece counting result is improved.
The embodiments of the present application will be further explained with reference to the drawings.
As shown in fig. 1, fig. 1 is a flowchart illustrating steps of a FIDT-based workpiece counting method according to an embodiment of the present application, and the embodiment of the present application provides a FIDT-based workpiece counting method including, but not limited to, the following steps:
step S110, acquiring a target image, and establishing an FIDT (Fidt specification) according to the target image, wherein the target image comprises a plurality of workpiece images;
step S120, a plurality of target local maximum values are obtained from the FIDT according to a preset local maximum value detection strategy, and target coordinate information corresponding to each target local maximum value is determined, wherein each target coordinate information represents coordinate information of the center position of each workpiece image;
step S130, calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm;
step S140, generating each bounding box according to the size of each instance;
and S150, carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result.
It should be noted that the FIDT-based workpiece counting method provided in the embodiment of the present application may be applied to a background server, and the embodiment of the present application does not limit a specific acquisition manner of a target image, for example, an image of an area to be identified in a production workshop is captured by a mobile device as the target image, and the target image is transmitted to the background server through a 5G network in real time; it will be appreciated that acquiring a target image comprising a plurality of workpiece images can provide an effective data basis for obtaining accurate workpiece count results.
It is understood that a Focal Inverse Distance Transform (FIDT) map created from the target image can indicate the density of the workpiece images, and is more dispersed than the density map, so that it is more convenient to locate each workpiece image; acquiring a preset local maximum value detection strategy, predicting a plurality of local maximum values in the FIDT through the local maximum value detection strategy, determining target coordinate information corresponding to each local maximum value according to the counting result of the workpiece images corresponding to the number of the local maximum values in the FIDT, and determining the central coordinate information corresponding to each workpiece image according to the target coordinate information; the method comprises the steps of calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm, generating each enclosure frame according to each example size, and carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result.
It should be noted that, the specific steps of extracting the multiple pieces of target coordinate information in the FIDT by the local maximum detection strategy in the present application may be as follows: the method comprises the steps of obtaining all local maximum values in the FIDT by utilizing a maximum pooling layer with a convolution kernel size of 3 x 3, wherein errors possibly exist in all the local maximum values (for example, a target image background is mistakenly regarded as a workpiece image), the local maximum values in the FIDT can be understood as pixel values, the pixel values which are true positive are far larger than the pixel values which are false positive, determining the pixel values which are true positive in the FIDT as the target local maximum values, filtering all the local maximum values by presetting a local maximum value threshold, filtering the pixel values which are false positive, obtaining available pixel values, namely obtaining the target local maximum values, and determining target coordinate information corresponding to the target local maximum values. The embodiment of the present application does not limit the specific value of the local maximum threshold, and those skilled in the art can determine the value according to actual situations.
In addition, before extracting the plurality of pieces of target coordinate information in the FIDT by the local maximum detection strategy, the workpiece counting method provided by the embodiment of the present application further includes: the loss calculation is performed on the FIDT, so that the quality of the FIDT is improved, and a specific formula of a loss function used in the embodiment of the present application is as follows:
L=L MSE +L 1-S
wherein L is MSE Is a loss of mean square error; l is 1-s For independent I-SSIM losses, in particular, L 1-s The formula of (1) is as follows:
Figure BDA0003747959180000041
wherein N is the total number of workpieces, E n And G n An estimate map and a true tag map for the nth independent instance area; the region size of each instance was set to 30 x 30 for all datasets; the formula for Ls is as follows:
L S =1-SSIM(E,G);
wherein, the specific formula of SSIM is as follows:
Figure BDA0003747959180000042
wherein E is an estimation graph; g is a real label graph; mu is a mean value; σ is the variance; lambda [ alpha ] 1 And λ 2 Respectively take 0.0001 and 0.0009 to obtainAvoid being divided by 0, the value range of SSIM is [ -1,1]When SSIM =1, it indicates that the estimated map is the same as the true tag map.
It can be understood that the SSIM loss function scans the entire FIDT with a sliding window, without distinguishing between foreground (where the workpiece image is) and background, while for the positioning task, counting by detecting local maxima, the structural loss should be concentrated in the local maxima area; global SSIM loss may generate high response, resulting in some false local maxima in the background, while I-SSIM loss makes it easier for the model to learn the independent region representation at the feature level, better identifying the local maxima, so that the mean square error loss and independent SSIM (I-SSIM) loss are proposed as the final loss function of the embodiments of the present application, which can guarantee the authenticity of the target local maxima obtained from FIDT.
It should be noted that, the preset clustering algorithm in the embodiment of the present application may be a KNN algorithm, and a specific formula is as follows:
Figure BDA0003747959180000051
wherein S (x, y) represents an example size of the workpiece image whose coordinate information is (x, y); p is the set of local maxima (i.e., predicted workpiece center positions);
Figure BDA0003747959180000052
constrained by a scalar factor f, which is the average distance between the coordinate information (x, y) and its k neighbors
Figure BDA0003747959180000053
The size of (d); the preset clustering algorithm can also be a K-means algorithm or a learning vectorization algorithm, and the skilled person can select the algorithm according to the actual situation, and the selection is not limited to a large extent.
In addition, referring to fig. 2, in an embodiment, the step S110 in the embodiment shown in fig. 1 further includes, but is not limited to, the following steps:
step S210, acquiring an image to be processed, and determining an interested area of the image to be processed, wherein the interested area is an area in which a plurality of workpiece images exist in the image to be processed;
and S220, cutting the workpiece image to be processed to obtain a target image, wherein the target image belongs to the region of interest.
It will be appreciated that the steps in the embodiment of fig. 1 for acquiring a target image comprising a plurality of workpiece images are: the interesting region of the image to be processed acquired by the mobile equipment is identified, the image to be processed is cut to obtain the target image belonging to the interesting region, and the image information (namely, the part not containing the workpiece image) of the non-interesting region in the image to be processed can be effectively removed, so that the acquisition efficiency of the workpiece counting result is effectively improved, and the accuracy of the workpiece counting result is improved.
In addition, referring to fig. 3, in an embodiment, before the step S110 of the embodiment shown in fig. 1 is executed to establish the FIDT according to the target image, the FIDT-based workpiece counting method provided by the present application further includes, but is not limited to, the following steps:
step S310, image preprocessing is carried out on the target image according to a preset preprocessing rule.
It can be understood that the main purpose of image preprocessing on the target image is to eliminate irrelevant information in the target image, recover useful real information, enhance the detectability of relevant information and simplify data to the maximum extent, thereby improving the reliability of subsequent target image application.
It should be noted that, in the embodiment of the present application, a specific method for performing image preprocessing on a target image is not limited, and may be performing image random number rotation processing, normalization processing, image binarization processing, or the like on the target image, and a person skilled in the art may select a specific mode of image preprocessing according to actual situations.
In addition, in an embodiment, the target image further includes annotation points corresponding to the respective workpiece images, and referring to fig. 4, the step S110 in the embodiment shown in fig. 1 further includes, but is not limited to, the following steps:
step S410, acquiring pixel points of the target image, and calculating Euclidean distances between each pixel point and each annotation point;
step S420, the FIDT is established according to the Euclidean distances.
It can be understood that the specific formula for obtaining the pixel points of the target image is as follows:
Figure BDA0003747959180000061
b is a set of all the marking points in the target image; (x, y) is any pixel point in the target image; (x ', y') is an annotation point in the target image; p (x, y) is the Euclidean distance between each pixel point and each label point; the specific formula for establishing FIDT from Euclidean distance is as follows:
Figure BDA0003747959180000062
wherein I is FIDT; the value of α is 0.02; the value of β is 0.75; c is an extra constant (let C = 1) to avoid division by 0.
In addition, in an embodiment, the target image further includes annotation points corresponding to the respective workpiece images, and referring to fig. 5, the step S150 in the embodiment shown in fig. 1 further includes, but is not limited to, the following steps:
step S510, acquiring a first image, wherein the first image is an image comprising each surrounding frame, and the size of the first image is the same as that of a target image;
step S520, attaching masks to all bounding boxes in the first image to obtain a second image;
step S530, inputting the second image and the target image into a preset object detector for iterative detection processing to obtain a first workpiece counting result;
in step S540, when the preset condition is satisfied, the sum of all the first workpiece counting results is determined as the target workpiece counting result.
It can be understood that the object detector provided in this embodiment of the present application is configured to attach a mask to all bounding boxes in a history feature image (corresponding to a first image) generated after a previous iteration process, re-establish a new FIDT on the history feature image (corresponding to a second image) to which the mask is attached, perform local maximum detection processing on the new FIDT to obtain a plurality of new target local maximum values of the new FIDT, determine new target coordinate information corresponding to each new target local maximum value, calculate an instance size of each workpiece image corresponding to each new target coordinate information according to a preset clustering algorithm, generate a new bounding box according to each instance size, combine an image including the new bounding box with the target image to obtain a new history feature image, attach a mask to the bounding boxes in the new history feature image, perform iteration operations of the above steps on the new history feature image to which the mask is attached again until a preset condition is satisfied, and end the iteration detection processing step; it can be understood that, in the embodiment of the present application, the number of bounding boxes corresponding to the historical feature image generated in each iteration step is recorded, that is, the first workpiece counting result obtained after the iteration detection processing step is completed in each iteration step is recorded, and when the iteration process is finished, the sum of all the first workpiece counting results is determined as the target workpiece counting result. It should be noted that the preset condition in the embodiment of the present application may include one of the following: the iteration times of the object detector for carrying out iteration detection processing on the second image and the target image exceed a preset iteration time threshold; alternatively, the first workpiece count result is 0.
In an embodiment, a specific formula of the object detector performing the iterative detection processing on the second image and the target image is as follows:
Figure BDA0003747959180000063
wherein the content of the first and second substances,
Figure BDA0003747959180000064
b is a surrounding frame combination; w is the width of each bounding box; h is the height of each enclosure frame; h is belonged to Z w×h For each iteration, the historical feature image (corresponding to the first image) input to the object detectorEach pixel of the feature image contains a number of detected boxes covering the pixel; and I is a target image.
Based on the above formula and with reference to the description of the above embodiment, the iterative process can be expressed as: giving a target image I, sequentially establishing FIDT according to the target image I, carrying out local maximum detection processing and generating a set of surrounding frames B, and during the first iteration, obtaining a historical characteristic image H 1 Empty, the object detector will input images I and H 1 Mapping to a set of bounding boxes B 1 Will comprise an enclosure frame B 1 Is recorded as a new history feature image H 2 (ii) a On the second iteration, H 2 Mapping to a target image I, attaching a mask, and then an object detector combines the input image I and the masked H 2 Mapping to a new set of bounding boxes B 2 Will contain an enclosure frame B 2 Is recorded as a new history feature image H 3 Containment set | B when the number of iterations reaches a limit or during the m-th iteration m When | =0, that is, a new bounding box is not detected in the mth iteration process, the iteration process is ended, and finally, a bounding box complete set obtained in each iteration process is output, wherein a specific formula is as follows:
Figure BDA0003747959180000071
wherein m represents the total number of iterations; b t The set of bounding boxes predicted for the object detector for each iteration.
The embodiments of the present application
In addition, in the iterative process, the technical scheme provided by the embodiment of the application further comprises the following steps: randomly dividing the bounding box set B into two subsets B old And B new ,B old And B new The relationship between them is expressed as:
Figure BDA0003747959180000072
B old ∪B new =φ;
b is to be old Maps to historical feature image H and forces the object detector to predict the missing bounding box B in H new Predicting the box B and the target box B by back propagation new While optimizing the loss of the object detector by the error between B and B old And B new And the effect of data enhancement is achieved.
In addition, after obtaining the counting result of the target workpiece, the technical solution of the embodiment of the present application further includes: and returning the target algorithm result to the front-end equipment for displaying, thereby providing an effective data basis for subsequent statistics, correction and historical record work.
In addition, referring to fig. 6, in an embodiment, the present application provides an FIDT-based workpiece counting apparatus 600, where the FIDT-based workpiece counting apparatus 600 includes:
the FIDT establishing module 610 and the FIDT establishing module 610 are used for acquiring a target image and establishing the FIDT according to the target image, wherein the target image comprises a plurality of workpiece images;
the target coordinate information acquiring module 620 is configured to acquire a preset local maximum detection strategy, acquire a plurality of target local maxima from the FIDT according to the local maximum detection strategy, and determine target coordinate information corresponding to each target local maximum, where each target coordinate information represents coordinate information of a center position of each workpiece image;
the example size calculation module 630, the example size calculation module 630 is configured to calculate, according to a preset clustering algorithm, an example size of each workpiece image corresponding to each target coordinate information;
the bounding box generating module 640, wherein the bounding box generating module 640 is configured to generate each bounding box according to each instance size;
a workpiece counting result obtaining module 650, wherein the workpiece counting result obtaining module 650 is configured to perform target detection processing on each bounding box and the target image according to a preset iterative object detection algorithm, so as to obtain a target workpiece counting result.
In addition, referring to fig. 7, fig. 7 is a structural diagram of an FIDT-based workpiece counting apparatus according to another embodiment of the present application, and an embodiment of the present application further provides an FIDT-based workpiece counting apparatus 700, where the FIDT-based workpiece counting apparatus 700 includes: memory 710, processor 720, and computer programs stored on memory 710 and operable on processor 720.
The processor 720 and the memory 710 may be connected by a bus or other means.
Non-transitory software programs and instructions required to implement the FIDT-based workpiece counting method of the above-described embodiment are stored in the memory 710, and when executed by the processor 720, perform the FIDT-based workpiece counting method of the above-described embodiment, for example, performing the above-described method steps S110 to S150 in fig. 1, S210 to S220 in fig. 2, S310 in fig. 3, S410 to S420 in fig. 4, and S510 to S540 in fig. 5.
The above-described embodiments of the apparatus are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may also be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Furthermore, an embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions, which are executed by a processor or controller, for example, by a processor 720 in the above-mentioned embodiment of the FIDT-based workpiece counting apparatus 700, and can make the processor 720 execute the FIDT-based workpiece counting method in the above-mentioned embodiment, for example, execute the above-mentioned method steps S110 to S150 in fig. 1, method steps S210 to S220 in fig. 2, method step S310 in fig. 3, method steps S410 to S420 in fig. 4, and method steps S510 to S540 in fig. 5. One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

Claims (10)

1. A workpiece counting method based on a focus back distance map FIDT is characterized by comprising the following steps:
acquiring a target image, and establishing an FIDT (Fidt image data set) according to the target image, wherein the target image comprises a plurality of workpiece images;
acquiring a plurality of target local maximum values from the FIDT according to a preset local maximum value detection strategy, and determining target coordinate information corresponding to each target local maximum value, wherein each target coordinate information represents coordinate information of the center position of each workpiece image;
calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm;
generating each bounding box according to each instance size;
and carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result.
2. The method of claim 1, wherein the acquiring a target image comprises:
acquiring an image to be processed, and determining an interested area of the image to be processed, wherein the interested area is an area in which a plurality of workpiece images exist in the image to be processed;
and cutting the workpiece image to be processed to obtain the target image, wherein the target image belongs to the region of interest.
3. The method of claim 1, wherein prior to said establishing FIDT from said target image, said method further comprises:
and carrying out image preprocessing on the target image according to a preset preprocessing rule.
4. The method of claim 1, wherein the target image further comprises annotation points corresponding to respective workpiece images; the establishing of the FIDT according to the target image comprises the following steps:
acquiring pixel points of the target image, and calculating Euclidean distances between the pixel points and the labeling points;
and establishing the FIDT according to the Euclidean distances.
5. The method of claim 1, wherein the performing target detection processing on each of the bounding box and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result comprises:
acquiring a first image, wherein the first image is an image comprising each surrounding frame, and the first image and the target image have the same size;
attaching a mask to all the bounding boxes in the first image to obtain a second image;
inputting the second image and the target image to a preset object detector for iterative detection processing to obtain a first workpiece counting result;
and when a preset condition is met, determining the sum of all the first workpiece counting results as the target workpiece counting result.
6. The method of claim 5, wherein the preset condition comprises at least one of:
the iteration times of the object detector for carrying out iteration detection processing on the second image and the target image exceed a preset iteration time threshold;
alternatively, the first and second electrodes may be,
the first workpiece count result is 0.
7. The method according to claim 1, characterized in that the clustering algorithm is a nearest neighbor rule classification, KNN, algorithm.
8. An FIDT-based workpiece counting apparatus, comprising:
the FIDT establishing module is used for acquiring a target image and establishing the FIDT according to the target image, wherein the target image comprises a plurality of workpiece images;
the target coordinate information acquisition module is used for acquiring a plurality of target local maximum values from the FIDT according to a preset local maximum value detection strategy and determining target coordinate information corresponding to each target local maximum value, wherein each piece of target coordinate information represents coordinate information of the center position of each workpiece image;
the example size calculation module is used for calculating the example size of each workpiece image corresponding to each target coordinate information according to a preset clustering algorithm;
a bounding box generation module, configured to generate each bounding box according to each instance size;
and the workpiece counting result acquisition module is used for carrying out target detection processing on each enclosure frame and the target image according to a preset iterative object detection algorithm to obtain a target workpiece counting result.
9. An FIDT-based workpiece counting apparatus comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the FIDT based workpiece counting method according to any of claims 1 to 7.
10. A computer readable storage medium storing computer executable instructions for performing the FIDT-based workpiece counting method according to any of claims 1 to 7.
CN202210830288.XA 2022-07-15 2022-07-15 Workpiece counting method and device based on FIDT and computer readable storage medium Pending CN115330687A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210830288.XA CN115330687A (en) 2022-07-15 2022-07-15 Workpiece counting method and device based on FIDT and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210830288.XA CN115330687A (en) 2022-07-15 2022-07-15 Workpiece counting method and device based on FIDT and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN115330687A true CN115330687A (en) 2022-11-11

Family

ID=83918393

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210830288.XA Pending CN115330687A (en) 2022-07-15 2022-07-15 Workpiece counting method and device based on FIDT and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN115330687A (en)

Similar Documents

Publication Publication Date Title
CN108388896B (en) License plate identification method based on dynamic time sequence convolution neural network
CN109118473B (en) Angular point detection method based on neural network, storage medium and image processing system
CN109934847B (en) Method and device for estimating posture of weak texture three-dimensional object
CN107784288B (en) Iterative positioning type face detection method based on deep neural network
CN113012157B (en) Visual detection method and system for equipment defects
JP5939056B2 (en) Method and apparatus for positioning a text region in an image
CN111639653B (en) False detection image determining method, device, equipment and medium
CN111275040B (en) Positioning method and device, electronic equipment and computer readable storage medium
CN113111716B (en) Remote sensing image semiautomatic labeling method and device based on deep learning
CN111523463B (en) Target tracking method and training method based on matching-regression network
CN113850136A (en) Yolov5 and BCNN-based vehicle orientation identification method and system
CN115527050A (en) Image feature matching method, computer device and readable storage medium
CN110334775B (en) Unmanned aerial vehicle line fault identification method and device based on width learning
CN115187879A (en) Building detection and floor positioning method and device based on YOLOv5
CN113421210A (en) Surface point cloud reconstruction method based on binocular stereo vision
CN116977895A (en) Stain detection method and device for universal camera lens and computer equipment
CN115330687A (en) Workpiece counting method and device based on FIDT and computer readable storage medium
CN115830310A (en) Workpiece point cloud segmentation method, device and medium based on channel attention mechanism
AU2020272936B2 (en) Methods and systems for crack detection using a fully convolutional network
Williams et al. Detecting marine animals in underwater video: Let's start with salmon
CN111723737A (en) Target detection method based on multi-scale matching strategy deep feature learning
CN115984316B (en) Industrial image edge extraction method and device for complex environment
CN115049731B (en) Visual image construction and positioning method based on binocular camera
US12039441B2 (en) Methods and systems for crack detection using a fully convolutional network
CN115700814A (en) Parking space detection method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination