CN117612061A - Visual detection method for package stacking state for stacking separation - Google Patents
Visual detection method for package stacking state for stacking separation Download PDFInfo
- Publication number
- CN117612061A CN117612061A CN202311491890.6A CN202311491890A CN117612061A CN 117612061 A CN117612061 A CN 117612061A CN 202311491890 A CN202311491890 A CN 202311491890A CN 117612061 A CN117612061 A CN 117612061A
- Authority
- CN
- China
- Prior art keywords
- package
- image
- lightweight
- model
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 71
- 238000000926 separation method Methods 0.000 title claims abstract description 49
- 230000000007 visual effect Effects 0.000 title abstract description 10
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000012549 training Methods 0.000 claims description 46
- 230000011218 segmentation Effects 0.000 claims description 39
- 238000011179 visual inspection Methods 0.000 claims description 16
- 230000007246 mechanism Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 238000003709 image segmentation Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 6
- 238000005286 illumination Methods 0.000 abstract description 2
- 230000002349 favourable effect Effects 0.000 abstract 1
- 238000012795 verification Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- 235000012766 Cannabis sativa ssp. sativa var. sativa Nutrition 0.000 description 1
- 235000012765 Cannabis sativa ssp. sativa var. spontanea Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000009120 camo Nutrition 0.000 description 1
- 235000005607 chanvre indien Nutrition 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 239000011487 hemp Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a visual detection method for package stacking state for stack separation, which relates to the technical field of logistics. The method can also have more accurate detection results and high detection speed under the influence of complex environment and illumination, is favorable for optimizing the stack separation effect, is easy to deploy a lightweight model, and is very suitable for the field use of a full-automatic bag supply system.
Description
Technical Field
The application relates to the technical field of logistics, in particular to a visual detection method for package stacking state for stacking separation.
Background
Along with the rapid development of the express logistics industry, the parcel sorting quantity is explosive growth, and the traditional mode of manually carrying out parcel supply and sorting is low in efficiency, so that the current requirements cannot be met.
In order to improve package sorting efficiency, at present, the logistics industry can use a full-automatic package supply system to realize automatic package supply and sorting, but packages entering the full-automatic package supply system are randomly placed in an initial state, and often have stacking conditions between the packages, so that sorting is not facilitated, and the first link of the full-automatic package supply system is to utilize a stacking separation mechanism to stack and separate packages stacked mutually, so that the later sorting efficiency is improved. The accurate detection of the stacking state between packages is a precondition for achieving a better stacking separation effect, but the stacking separation effect is affected because the use environment of a full-automatic package supply system is complex and the stacking state between packages cannot be accurately and effectively detected at present.
Disclosure of Invention
The application provides a visual detection method for package stacking state of stack separation aiming at the problems and the technical requirements, and the technical scheme of the application is as follows:
a visual inspection method of package stacking status for stack separation, the visual inspection method of package stacking status comprising:
acquiring an original image of a separation section conveyor belt of the stacking separation mechanism through image acquisition equipment, wherein the original image comprises the separation section conveyor belt and packages transmitted on the separation section conveyor belt;
performing target detection on the original image by using a lightweight target detection model based on the PP-Picode to obtain a prediction frame of each package included in the original image;
image segmentation is carried out on the image at the prediction frame of each package by utilizing a lightweight semantic segmentation model based on the PP-mobileseg, so that a prediction mask of each package is obtained;
the area ratio of the sum of the pixel areas of the prediction masks of all packages to the sum of the pixel areas of the separation section conveyor belt is calculated, and the stacking state of the packages on the separation section conveyor belt is determined according to the area ratio.
The lightweight target detection model based on the PP-Picodet comprises a main network, a connecting network and a head network which are sequentially connected, wherein the main network adopts Enhanced ShuffleNet, the connecting network adds downsampling once on CSP-PAN, and the head network adopts PicoHeadV2;
the lightweight semantic segmentation model based on the PP-mobileseg comprises a StrideFormer module, a feature fusion block AAM and an upsampling module VIM.
The further technical scheme is that the method for determining the stacking state of the packages on the separated section conveying belt according to the area ratio comprises the following steps:
when the area ratio S is more than or equal to S max Determining that the packages on the separator conveyor belt are seriously stacked;
when the area is occupied by S min <S<S max Determining a stack of wrapped portions on the separator conveyor belt;
when the area ratio S is less than or equal to S min Determining that there is no stacking of packages on the separator conveyor belt;
wherein S is min And S is max Parameters are respectively.
The further technical scheme is that the method for obtaining the prediction frame of each package included in the original image comprises the following steps:
extracting an image of a region where the separation section conveyor belt is located from an original image as an image of a region of interest, and inputting the image of the region of interest into a lightweight target detection model to obtain a plurality of candidate frames corresponding to each package and corresponding confidence;
and removing the candidate frames with the cross ratio larger than a preset threshold value by using a non-maximum suppression method based on the confidence coefficient of the candidate frames of the packages to obtain a prediction frame of each package.
The further technical scheme is that the image segmentation of the image at the prediction frame of each package by using the lightweight semantic segmentation model based on the PP-mobileseg comprises the following steps:
and (3) expanding the width and the height of the prediction frame of each package to obtain a package region of the package, cutting the image of the package region of each package from the original image, and inputting a lightweight semantic segmentation model based on the PP-mobileseg for image segmentation.
The visual detection method for the package stacking state further comprises the following steps:
constructing a network structure of a lightweight target detection model based on the PP-Picode;
randomly initializing network parameters of a lightweight target detection model, performing model pre-training based on the initialized network parameters by using a COCO public data set, and taking the weight of the optimal MAP obtained by model pre-training as a basic network parameter of the lightweight target detection model;
carrying out model training on the lightweight target detection model based on basic network parameters by using the parcel detection data set, and taking the weight of F1-score obtained by model training as the network parameters obtained by training to obtain a lightweight target detection model after training; the model is trained by using a SimOTA sampling strategy, a Cosine learning rate attenuation strategy and an H-Swish activation function.
The visual detection method for the package stacking state further comprises the following steps:
constructing a network structure of a lightweight semantic segmentation model based on PP-mobileseg;
randomly initializing network parameters of a lightweight semantic segmentation model, performing model pre-training based on the initialized network parameters by utilizing a cityscape public data set, and taking the weight of an optimal MAP obtained by model pre-training as a basic network parameter of the lightweight semantic segmentation model;
carrying out model training on the lightweight semantic segmentation model based on basic network parameters by using the parcel segmentation data set, and taking the weight of F1-score obtained by model training as the network parameters obtained by training to obtain a trained lightweight semantic segmentation model; the model training uses Adam learning rate strategy and polynomialaddeay learning rate decay strategy.
The visual detection method for the package stacking state further comprises the following steps:
acquiring a sample package image of a separation section conveyor belt of the stack separation mechanism;
cutting out an image of the area where the separating section conveyor belt is located from each sample package image, and marking a minimum circumscribed positive rectangular frame of the package as a prediction frame to obtain a detection sample image;
carrying out data enhancement processing on all the detection sample images, and constructing a package detection data set; wherein the data enhancement processing includes at least one of image rotation, brightness transformation, and contrast transformation.
The visual detection method for the package stacking state further comprises the following steps:
cutting out a local image in a prediction frame of each package label from each detection sample image in the package detection data set, and marking a mask area of the package in the local image to obtain a segmentation sample image;
carrying out data enhancement processing on all the segmented sample images, and constructing a package segmented data set; wherein the data enhancement processing includes at least one of image rotation, brightness transformation, and contrast transformation.
The visual detection method for the package stacking state further comprises the following steps:
the lightweight object detection model and the lightweight semantic segmentation model are deployed by using a side-end deployment tool, namely the compact-lite, and are optimized by using an opt tool provided by the side-end deployment tool, namely the compact-lite.
The beneficial technical effects of this application are:
the method is characterized in that a pre-trained lightweight target detection model and a lightweight semantic segmentation model are utilized to accurately extract the sum of pixel areas, so that the area occupation ratio can be accurately obtained to serve as a reference basis for stack separation, the stack separation effect is favorably optimized, the method can also have more accurate detection results under the influence of complex environment and illumination, the detection speed is high, the lightweight model is easy to deploy, and the method is very suitable for the field use of a full-automatic package supply system.
Drawings
FIG. 1 is a method flow diagram of a visual inspection method of package stacking status according to one embodiment of the present application.
FIG. 2 is a flow chart of a method for training a lightweight object detection model and a lightweight semantic segmentation model in one embodiment of the present application.
Detailed Description
The following describes the embodiments of the present application further with reference to the accompanying drawings.
The application discloses a visual inspection method for package stacking state of stack separation, please refer to a flowchart shown in fig. 1, the visual inspection method for package stacking state comprises:
step 1, acquiring an original image of a separation section conveyor belt of a stack separation mechanism through image acquisition equipment.
The folding piece separating mechanism comprises a plurality of sections of conveying belts and at least comprises a separating section conveying belt, folding piece separating mechanisms can realize folding piece separation by adjusting the speed difference of front and rear belts of the separating section conveying belt, except for the separating section conveying belt, the folding piece separating mechanisms can also comprise conveying belts of a divergent section, a temporary storage section, a transition section and the like, and the folding piece separating mechanisms are existing mechanisms of the existing full-automatic bag supplying system and can refer to the existing actual structure and working principle.
The original image obtained includes a separate conveyor belt and packages transported thereon. In practice, the image acquisition device is typically mounted above and vertically toward the separator conveyor belt. In order to ensure that the problem that small packages are blocked by high packages does not occur, the image acquisition equipment needs to be arranged above the separation section conveying belt by more than 1 m. Since the separator conveyor belt is typically an inclined ramp-up section, it is ensured that the inclination of the image acquisition device is parallel to the belt surface of the separator conveyor belt.
In one embodiment, the image acquisition device adopts an area array CMOS image device, so that high-definition original images can still be captured under the low-light environment of a separation section conveyor belt running at high speed and an application site, and the resolution of the obtained original images can be 200 ten thousand pixels (1600×1200). When the device specifically works, the FPGA triggers the image acquisition device to acquire an image, and the actual trigger frame rate is required to be determined according to the highest control efficiency of the belt of the separation section, which can reach 30 frames/s at most.
And 2, performing target detection on the original image by using a pre-trained lightweight target detection model based on the PP-Picodet to obtain a prediction frame of each package included in the original image.
In one embodiment, the acquired original image typically further includes a background image, so that the image of the area where the separator conveyor belt is located is first extracted from the original image as an area of interest image, and then the area of interest image is input into a lightweight object detection model to obtain a prediction frame for each package included in the original image.
In addition, when the lightweight target detection model is used for detecting the prediction frames, a plurality of candidate frames corresponding to each package and corresponding confidence coefficients are obtained first. And then, rejecting the candidate frames with the cross ratio larger than a preset threshold value by using a non-maximum suppression method based on the confidence coefficient of the candidate frames of the packages, so that the optimal prediction frame of each package can be obtained. In practical application, the confidence coefficient threshold value can be set to be 0.5, and the cross ratio threshold value can be set to be 0.7.
And 3, performing image segmentation on the image at the prediction frame of each package by using a lightweight semantic segmentation model based on the PP-mobileseg to obtain a prediction mask of each package.
In another embodiment, the width and the height of the prediction frame of each package are enlarged to obtain package areas of the package, an image of the package areas of each package is cut out from an original image, and a lightweight semantic segmentation model based on PP-mobileseg is input for image segmentation. That is, the image of the prediction frame of the package is not directly intercepted as the package area, but is intercepted after the prediction frame is enlarged, so that the problem of inaccurate segmentation result caused by small prediction frame can be avoided. In practical application, the height and width of the prediction frame can be enlarged to 1.2 times to obtain the wrapping area.
And 4, calculating the area ratio of the sum of the pixel areas of all the wrapped predictive masks to the sum of the pixel areas of the separation section conveyor belt, and determining the stacking state of the wraps on the separation section conveyor belt according to the area ratio.
In the implementation, a single-channel image with the same size as the image of the region of interest in the original image and full black pixels is created, and then each wrapped binarized prediction mask is copied into the single-channel image, and pixels of the prediction mask part are white, so that a mask binary image is obtained. And calculating the area of all connected areas in the mask binary image to obtain the sum of the pixel areas of all wrapped prediction masks. The whole area of the mask binary image is calculated to be the sum of the pixel areas of the conveyor belts of the separation sections, so that the area occupation ratio can be calculated.
The stacking state of the packages on the separation section conveying belt can be determined according to the value range of the area ratio, and the stacking state of the packages on the separation section conveying belt is generally divided into three types according to the separation process of a common stacking separation mechanism: the package is seriously stacked, the package part is stacked and the package does not exist, and then the front belt and the rear belt of the separation section conveying belt are controlled to carry out corresponding differential speed adjustment according to different stacking states. Then when the area ratio S is greater than or equal to S max When the packages on the separator conveyor belt are severely stacked, the packages are determined. When the area is occupied by S min <S<S max When a stack of wrapped portions on the separator conveyor belt is determined. When the area ratio S is less than or equal to S min When it is determined that there is no stacking of packages on the separator conveyor belt. Wherein S is max And S is min Is a self-defined duty cycle threshold parameter, for example, S can be set min ＝20％，S max =80%. The stacking state of packages characterized by specific different area ratios and the corresponding differential speed regulating mechanism adopted are presetAs set forth herein, this application is not limited thereto.
The application needs to use a pre-trained lightweight target detection model based on the PP-Picodet and a lightweight semantic segmentation model based on the PP-mobileseg, use a side-to-side tool pad-lite to deploy the two models, and optimize the two models by using an opt tool provided by the side-to-side tool pad-lite, so that less resources can be consumed on the side to achieve faster execution speed, and the application requirements of an industrial field are met. The training process for the two models is also included before use, and is described below with reference to fig. 2:
1. training of PP-Picodet-based lightweight target detection model
(1) And constructing a network structure of a lightweight target detection model based on the PP-Picodet.
The built lightweight target detection model based on the PP-Picodet comprises a main network, a connecting network and a head network which are sequentially connected, wherein the main network adopts Enhanced ShuffleNet, a SE module and a Ghost module in the Ghost Net are introduced on the basis of a SheffeNetV 2, depth separable convolution is added, and different channel information is fused to improve model accuracy. The connection network is added with downsampling once on CSP-PAN, and a smaller characteristic scale is added to improve the detection effect of oversized packages such as hemp bags and the like. The header network adopts PicoHeadV2;
(2) Randomly initializing network parameters of the lightweight target detection model, performing model pre-training based on the initialized network parameters by using the COCO public data set, and taking the weight of the optimal MAP obtained by model pre-training as the basic network parameters of the lightweight target detection model. The number of iterations is typically 160000.
(3) And carrying out model training on the lightweight target detection model based on the basic network parameters by using the parcel detection data set, and taking the weight of F1-score obtained by model training as the network parameters obtained by training, thereby obtaining the trained lightweight target detection model.
When the model is trained by using the package detection data set, a SimOTA sampling strategy is used, a label distribution mode is dynamically changed along with the training process, and the model convergence is effectively accelerated by collecting high-quality samples in a target area. Using a more excellent Cosine learning rate decay strategy, an H-Swish activation function was used instead of the general Relu activation function. Typically 300 epochs are used for the training step.
The package detection data set used in the method needs to be prefabricated, and comprises the steps of obtaining sample package images of the separation section conveyor belt of the stacking separation mechanism, then cutting out the images of the area where the separation section conveyor belt is located from each sample package image, and marking the minimum external positive rectangular frame of each package as a corresponding prediction frame by using a labelme marking tool to obtain a detection sample image. And carrying out data enhancement processing on all the detection sample images to expand the data set, and constructing and obtaining the package detection data set. Wherein the data enhancement processing includes at least one of image rotation, brightness transformation, and contrast transformation. The constructed package detection data set is divided into a training set and a verification set according to the ratio of 8:2 for model training and verification.
2. Training of PP-mobileseg-based lightweight semantic segmentation model
(1) And constructing a network structure of a lightweight semantic segmentation model based on the PP-mobileseg. The lightweight semantic segmentation model based on the PP-mobileseg comprises a StrideFormer module, a feature fusion block AAM and an upsampling module VIM.
(2) Randomly initializing network parameters of the lightweight semantic segmentation model, performing model pre-training based on the initialized network parameters by utilizing a cityscape public data set, and taking the weight of the optimal MAP obtained by model pre-training as the basic network parameters of the lightweight semantic segmentation model.
(3) And carrying out model training on the lightweight semantic segmentation model based on the basic network parameters by using the parcel segmentation data set, and taking the weight of F1-score obtained by model training as the network parameters obtained by training to obtain the trained lightweight semantic segmentation model. The Adam learning rate strategy and PolynomialDecay learning rate decay strategy are used in model training, and the training step length is generally 300 epochs.
Also, there is a need to pre-make a parcel split dataset, including: and cutting out a local image in a prediction frame marked by each package from the detected sample image for each detected sample image in the package detection data set, and marking a mask area of the package in the local image to obtain a segmented sample image. And then carrying out data enhancement processing on all the segmented sample images to expand the data set, and constructing to obtain the wrapped segmented data set. Wherein the data enhancement processing includes at least one of image rotation, brightness transformation, and contrast transformation. Likewise, the constructed parcel segmentation dataset is divided into a training set and a verification set according to the ratio of 8:2 for model training and verification.
The above are only preferred embodiments of the present application, and the present application is not limited to the above examples. It is to be understood that other modifications and variations which may be directly derived or contemplated by those skilled in the art without departing from the spirit and concepts of the present application are to be considered as being included within the scope of the present application.
Claims (10)
1. A visual inspection method for package stacking status for stack separation, characterized in that the visual inspection method for package stacking status comprises:
acquiring an original image of a separation section conveyor belt of a stacking separation mechanism through image acquisition equipment, wherein the original image comprises the separation section conveyor belt and packages transmitted on the separation section conveyor belt;
performing target detection on the original image by using a lightweight target detection model based on the PP-Picode to obtain a prediction frame of each package included in the original image;
image segmentation is carried out on the image at the prediction frame of each package by utilizing a lightweight semantic segmentation model based on the PP-mobileseg, so that a prediction mask of each package is obtained;
and calculating the area ratio of the sum of the pixel areas of the prediction masks of all packages to the sum of the pixel areas of the separation section conveyor belt, and determining the stacking state of the packages on the separation section conveyor belt according to the area ratio.
2. The visual inspection method of package stacking status of claim 1,
the lightweight target detection model based on the PP-Picodet comprises a main network, a connecting network and a head network which are sequentially connected, wherein the main network adopts Enhanced ShuffleNet, the connecting network adds downsampling once on CSP-PAN, and the head network adopts PicoHeadV2;
the lightweight semantic segmentation model based on the PP-mobileseg comprises a StrideFormer module, a feature fusion block AAM and an upsampling module VIM.
3. The visual inspection method of package stacking status of claim 1, wherein said determining the stacking status of packages on separate conveyor belts based on said area ratio comprises:
when the area ratio S is greater than or equal to S max Determining that the packages on the separator conveyor belt are seriously stacked;
when the area ratio S min <S<S max Determining a stack of wrapped portions on the separator conveyor belt;
when the area ratio S is less than or equal to S min Determining that there is no stacking of packages on the separator conveyor belt;
wherein S is min And S is max Parameters are respectively.
4. The package-on-package visual inspection method of claim 1, wherein the deriving a predicted box for each package included in the original image comprises:
extracting an image of a region where the separation section conveyor belt is located from the original image as an image of a region of interest, and inputting the image of the region of interest into the lightweight object detection model to obtain a plurality of candidate frames corresponding to each package and corresponding confidence levels;
and removing the candidate frames with the cross ratio larger than a preset threshold value by using a non-maximum suppression method based on the confidence coefficient of the candidate frames of the packages to obtain a prediction frame of each package.
5. The visual inspection method of package stacking status according to claim 1, wherein said image segmentation of the image at the prediction box of each package using a PP-mobileseg based lightweight semantic segmentation model comprises:
and expanding the width and the height of the prediction frame of each package to obtain a package region of the package, cutting an image of the package region of each package from the original image, and inputting a lightweight semantic segmentation model based on PP-mobileseg to carry out image segmentation.
6. The visual inspection method of package stacking status of claim 1, further comprising:
constructing a network structure of a lightweight target detection model based on the PP-Picode;
randomly initializing network parameters of a lightweight target detection model, performing model pre-training based on the initialized network parameters by using a COCO public data set, and taking the weight of the optimal MAP obtained by model pre-training as a basic network parameter of the lightweight target detection model;
carrying out model training on the lightweight target detection model based on basic network parameters by using the parcel detection data set, and taking the weight of F1-score obtained by model training as the network parameters obtained by training to obtain a lightweight target detection model after training; the model is trained by using a SimOTA sampling strategy, a Cosine learning rate attenuation strategy and an H-Swish activation function.
7. The visual inspection method of package stacking status of claim 6, further comprising:
constructing a network structure of a lightweight semantic segmentation model based on PP-mobileseg;
randomly initializing network parameters of a lightweight semantic segmentation model, performing model pre-training based on the initialized network parameters by utilizing a cityscape public data set, and taking the weight of an optimal MAP obtained by model pre-training as a basic network parameter of the lightweight semantic segmentation model;
carrying out model training on the lightweight semantic segmentation model based on basic network parameters by using the parcel segmentation data set, and taking the weight of F1-score obtained by model training as the network parameters obtained by training to obtain a trained lightweight semantic segmentation model; the model training uses Adam learning rate strategy and polynomialaddeay learning rate decay strategy.
8. The visual inspection method of package stacking status of claim 7, further comprising:
acquiring a sample package image of a separation section conveyor belt of the stack separation mechanism;
cutting out an image of the area where the separating section conveyor belt is located from each sample package image, and marking a minimum circumscribed positive rectangular frame of the package as a prediction frame to obtain a detection sample image;
carrying out data enhancement processing on all the detection sample images, and constructing and obtaining the package detection data set; wherein the data enhancement processing includes at least one of image rotation, brightness transformation, and contrast transformation.
9. The visual inspection method of package stacking status of claim 8, further comprising:
cutting out a local image in a prediction frame of each package label from each detection sample image for each detection sample image in the package detection data set, and marking a mask region of the package in the local image to obtain a segmentation sample image;
carrying out data enhancement processing on all the segmented sample images, and constructing to obtain the package segmented data set; wherein the data enhancement processing includes at least one of image rotation, brightness transformation, and contrast transformation.
10. The visual inspection method of package stacking status of claim 1, further comprising:
the lightweight object detection model and the lightweight semantic segmentation model are deployed by using a side-end deployment tool, namely the compact-lite, and are optimized by using an opt tool provided by the side-end deployment tool, namely the compact-lite.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311491890.6A CN117612061A (en) | 2023-11-09 | 2023-11-09 | Visual detection method for package stacking state for stacking separation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311491890.6A CN117612061A (en) | 2023-11-09 | 2023-11-09 | Visual detection method for package stacking state for stacking separation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117612061A true CN117612061A (en) | 2024-02-27 |
Family
ID=89950617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311491890.6A Pending CN117612061A (en) | 2023-11-09 | 2023-11-09 | Visual detection method for package stacking state for stacking separation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117612061A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118172716A (en) * | 2024-05-11 | 2024-06-11 | 中科微至科技股份有限公司 | Method for detecting package state of crossed belt trolley based on RGB-D image |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111814739A (en) * | 2020-07-27 | 2020-10-23 | 上海东普信息科技有限公司 | Method, device and equipment for detecting express package volume and storage medium |
| CN114120359A (en) * | 2021-11-15 | 2022-03-01 | 东南大学 | Method for measuring body size of group-fed pigs based on stacked hourglass network |
| CN114308680A (en) * | 2021-12-31 | 2022-04-12 | 中科微至智能制造科技江苏股份有限公司 | Control method for stack separation |
| CN114708208A (en) * | 2022-03-16 | 2022-07-05 | 杭州电子科技大学 | Famous tea tender shoot identification and picking point positioning method based on machine vision |
| CN114897816A (en) * | 2022-05-09 | 2022-08-12 | 安徽工业大学 | Mask R-CNN mineral particle identification and particle size detection method based on improved Mask |
| CN115690073A (en) * | 2022-11-11 | 2023-02-03 | 华南理工大学 | Local characterization method, device and medium for ceramic microstructure manufactured by laser additive manufacturing |
| CN116255912A (en) * | 2022-12-29 | 2023-06-13 | 江苏中科贯微自动化科技有限公司 | Method and system for measuring static volume of package |
| WO2023137871A1 (en) * | 2022-01-19 | 2023-07-27 | 福建南方路面机械股份有限公司 | Automatic sorting method and device based on height sensor and readable medium |
-
2023
- 2023-11-09 CN CN202311491890.6A patent/CN117612061A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111814739A (en) * | 2020-07-27 | 2020-10-23 | 上海东普信息科技有限公司 | Method, device and equipment for detecting express package volume and storage medium |
| CN114120359A (en) * | 2021-11-15 | 2022-03-01 | 东南大学 | Method for measuring body size of group-fed pigs based on stacked hourglass network |
| CN114308680A (en) * | 2021-12-31 | 2022-04-12 | 中科微至智能制造科技江苏股份有限公司 | Control method for stack separation |
| WO2023137871A1 (en) * | 2022-01-19 | 2023-07-27 | 福建南方路面机械股份有限公司 | Automatic sorting method and device based on height sensor and readable medium |
| CN114708208A (en) * | 2022-03-16 | 2022-07-05 | 杭州电子科技大学 | Famous tea tender shoot identification and picking point positioning method based on machine vision |
| CN114897816A (en) * | 2022-05-09 | 2022-08-12 | 安徽工业大学 | Mask R-CNN mineral particle identification and particle size detection method based on improved Mask |
| CN115690073A (en) * | 2022-11-11 | 2023-02-03 | 华南理工大学 | Local characterization method, device and medium for ceramic microstructure manufactured by laser additive manufacturing |
| CN116255912A (en) * | 2022-12-29 | 2023-06-13 | 江苏中科贯微自动化科技有限公司 | Method and system for measuring static volume of package |
Non-Patent Citations (2)
| Title |
|---|
| 叶舟: "PP-MobileSeg: 探索移动设备上又快又准的语义分割模型", pages 2 - 3, Retrieved from the Internet <URL:https://blog.csdn.net/oYeZhou/article/details/134112925> * |
| 百度大脑: "0.99M，150FPS，移动端超轻量目标检测算法PP-PicoDet来了！", pages 2 - 3, Retrieved from the Internet <URL:https://zhuanlan.zhihu.com/p/436625722> * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118172716A (en) * | 2024-05-11 | 2024-06-11 | 中科微至科技股份有限公司 | Method for detecting package state of crossed belt trolley based on RGB-D image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111126472B (en) | SSD (solid State disk) -based improved target detection method | |
| CN113052834B (en) | Pipeline defect detection method based on convolution neural network multi-scale features | |
| CN108492291B (en) | CNN segmentation-based solar photovoltaic silicon wafer defect detection system and method | |
| CN108647742A (en) | Fast target detection method based on lightweight neural network | |
| Ribeiro et al. | An end-to-end deep neural architecture for optical character verification and recognition in retail food packaging | |
| CN117612061A (en) | Visual detection method for package stacking state for stacking separation | |
| CN112233129B (en) | Deep learning-based parallel multi-scale attention mechanism semantic segmentation method and device | |
| CN113160062B (en) | Infrared image target detection method, device, equipment and storage medium | |
| CN113052006B (en) | Image target detection method, system and readable storage medium based on convolutional neural network | |
| CN109949209B (en) | Rope detection and removal method based on deep learning | |
| CN108710893A (en) | A kind of digital image cameras source model sorting technique of feature based fusion | |
| CN114494696A (en) | Method, system and device for rapidly detecting multi-scale coal gangue image | |
| CN113569981A (en) | Power inspection bird nest detection method based on single-stage target detection network | |
| CN114898200A (en) | Conveyor belt workpiece image target detection method and system based on light-weight YOLOV4-tiny | |
| CN107886065A (en) | A kind of Serial No. recognition methods of mixing script | |
| CN116385758A (en) | Detection method for damage to surface of conveyor belt based on YOLOv5 network | |
| CN112508029A (en) | Instance segmentation method based on target box labeling | |
| CN112418032A (en) | Human behavior recognition method and device, electronic equipment and storage medium | |
| CN113570540A (en) | Image tampering blind evidence obtaining method based on detection-segmentation architecture | |
| CN114359286A (en) | Insulator defect identification method, device and medium based on artificial intelligence | |
| CN112883964B (en) | Method for detecting characters in natural scene | |
| CN113901924A (en) | Document table detection method and device | |
| CN112541916A (en) | Waste plastic image segmentation method based on dense connection | |
| CN116958786A (en) | Dynamic visual identification method for chemical waste residues based on YOLOv5 and ResNet50 neural network | |
| CN116994161A (en) | Insulator defect detection method based on improved YOLOv5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |