CN114871115B - Object sorting method, device, equipment and storage medium - Google Patents
Object sorting method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114871115B CN114871115B CN202210460909.XA CN202210460909A CN114871115B CN 114871115 B CN114871115 B CN 114871115B CN 202210460909 A CN202210460909 A CN 202210460909A CN 114871115 B CN114871115 B CN 114871115B
- Authority
- CN
- China
- Prior art keywords
- identified
- image
- candidate frame
- target candidate
- sorting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000010586 diagram Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 5
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000004422 calculation algorithm Methods 0.000 abstract description 3
- 230000004913 activation Effects 0.000 description 19
- 238000010606 normalization Methods 0.000 description 19
- 230000006870 function Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an object sorting method, an object sorting device and a storage medium, wherein the method comprises the following steps: acquiring an image of an object to be identified; inputting an object image to be identified into a sorting network for object identification to obtain an identification result, extracting features of the sorting network according to the object image to be identified to obtain a feature image, predicting according to the feature image to obtain a center point of a target candidate frame, a bias value of the target candidate frame and the size of the target candidate frame, determining a target detection frame, and obtaining the identification result according to the target detection frame; sorting the objects according to the identification result; the sorting network reduces the complexity of network structure and network calculation, improves the detection performance, improves the running speed of an algorithm, improves the detection efficiency and has good robustness.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to a method, apparatus, device, and storage medium for sorting objects.
Background
Sorting objects is generally performed manually, which has the problems of high labor cost and low efficiency. At present, a method for identifying images of articles to be sorted through a neural network and controlling an execution manipulator to sort the articles to be sorted according to a sorting identification result is also available; the efficiency of this approach is largely determined by the performance of the neural network. When the partial neural network processes the object images containing a plurality of objects with high similarity, a plurality of objects which are mutually shielded, objects which are far away from the camera and objects which rotate at a high speed, the processing effect is poor, and the sorting efficiency is greatly influenced.
Disclosure of Invention
The invention aims to at least solve one of the technical problems in the prior art and provides an object sorting method, an object sorting device and a storage medium.
The invention solves the problems by adopting the following technical scheme:
In a first aspect of the invention, a method of sorting objects comprises:
Acquiring an image of an object to be identified;
inputting the object image to be identified into a sorting network for object identification to obtain an identification result, wherein the sorting network extracts characteristics according to the object image to be identified to obtain a characteristic diagram, predicts according to the characteristic diagram, outputs a center point of a target candidate frame, a bias value of the target candidate frame and the size of the target candidate frame respectively through three prediction branches, determines a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame and the size of the target candidate frame, and obtains the identification result according to the target detection frame;
and sorting the objects according to the identification result.
According to a first aspect of the invention, before the step of inputting the image of the item to be identified into the sorting network, it further comprises:
and adjusting the size of the image of the object to be identified through a size adjustment network.
According to a first aspect of the present invention, the resizing the image of the object to be identified through a resizing network comprises:
when the width of the object image to be identified is larger than a preset width, scaling the object image to be identified in the width direction to enable the width of the object image to be identified to be equal to the preset width;
when the height of the object image to be identified is larger than a preset height, scaling the object image to be identified in the high direction to enable the height of the object image to be identified to be equal to the preset height;
when the width of the object image to be identified is smaller than a preset width, carrying out zero padding treatment on the object image to be identified in the width direction so that the width of the object image to be identified is equal to the preset width;
When the height of the object image to be identified is smaller than a preset height, carrying out zero padding on the object image to be identified in the high direction, so that the height of the object image to be identified is equal to the preset height.
According to a first aspect of the present invention, predicting, based on the feature map, a center point of an output target candidate frame includes:
generating a thermodynamic diagram according to the characteristic diagram;
Scaling a target candidate frame into the thermodynamic diagram, and calculating center coordinates of a Gaussian circle corresponding to the target candidate frame;
calculating the radius of a Gaussian circle according to the size of the target candidate frame;
calculating a Gaussian value of the Gaussian circle according to the circle center coordinates and the radius;
And taking the position corresponding to the maximum value of the Gaussian value as the center point of the target candidate frame, and outputting the center point of the target candidate frame.
According to a first aspect of the present invention, predicting, based on the feature map, a bias value of a target pre-selected frame and a size of a target candidate frame are output, including:
performing maximum pooling treatment on a plurality of Gaussian values, sequencing the Gaussian values according to the numerical value from large to small, and taking all the Gaussian values ranked before a preset numerical value as target Gaussian values;
and taking the pixel point corresponding to the target Gaussian value as a target pixel point, and carrying out regression calculation according to the target pixel point to obtain the offset value of the target pre-selected frame and the size of the target candidate frame.
According to a first aspect of the present invention, after the step of extracting features from the image of the object to be identified to obtain a feature map, the method further includes:
And enlarging the size of the characteristic map through a deconvolution network.
According to the first aspect of the invention, the network structure adopted for extracting the characteristics according to the image of the object to be identified is Resnet-18 network structure.
In a second aspect of the present invention, an object sorting apparatus includes:
The image acquisition unit is used for acquiring an image of the object to be identified;
The image recognition unit is used for inputting the object image to be recognized into a sorting network to perform object recognition to obtain a recognition result, wherein the sorting network performs feature extraction according to the object image to be recognized to obtain a feature image, predicts according to the feature image, respectively outputs a center point of a target candidate frame, a bias value of the target candidate frame and the size of the target candidate frame through three prediction branches, determines a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame and the size of the target candidate frame, and obtains the recognition result according to the target detection frame;
And the sorting unit is used for sorting the objects according to the identification result.
In a third aspect of the present invention, an object sorting apparatus includes: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the object sorting method according to the first aspect of the invention when executing the computer program.
In a fourth aspect of the present invention, a storage medium stores a computer program for executing the object sorting method according to the first aspect of the present invention.
The scheme has at least the following beneficial effects: the sorting network is used for identifying the objects, so that the complexity of a network structure and the complexity of network calculation are reduced, the detection performance is improved, the running speed of an algorithm is improved, the detection efficiency is improved, the robustness is good, and the universality is good; thereby improving the sorting efficiency of the objects.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The invention is further described below with reference to the drawings and examples.
FIG. 1 is a flow chart of a method of sorting objects in accordance with an embodiment of the present invention;
FIG. 2 is a partial block diagram of a sorting network;
fig. 3 is a block diagram of an object sorting apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the accompanying drawings are used to supplement the description of the written description so that one can intuitively and intuitively understand each technical feature and overall technical scheme of the present invention, but not to limit the scope of the present invention.
In the description of the present invention, it should be understood that references to orientation descriptions such as upper, lower, front, rear, left, right, etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description of the present invention and to simplify the description, and do not indicate or imply that the apparatus or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present invention.
In the description of the present invention, a number means one or more, a number means two or more, and greater than, less than, exceeding, etc. are understood to not include the present number, and above, below, within, etc. are understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present invention can be reasonably determined by a person skilled in the art in combination with the specific contents of the technical scheme.
An embodiment of the first aspect of the present invention provides a method of sorting objects.
Referring to fig. 1, the object sorting method includes:
Step S100, acquiring an image of an object to be identified;
Step S200, inputting an article image to be identified into a sorting network for article identification to obtain an identification result, wherein the sorting network performs feature extraction according to the article image to be identified to obtain a feature image, predicts according to the feature image, outputs a center point of a target candidate frame, a bias value of the target candidate frame and a size of the target candidate frame respectively through three prediction branches, determines a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame and the size of the target candidate frame, and obtains the identification result according to the target detection frame;
and step S300, sorting objects according to the identification result.
Corresponding to step S100, the image of the object to be identified may be a still picture obtained by capturing a still image with a camera, or may be a moving video obtained by capturing a moving video with an image capturing apparatus, and then dividing the moving video.
After step S100, that is, before the image of the object to be identified is input to the sorting network for object identification, preprocessing, such as denoising or resizing, is required for the image of the object to be identified.
In this embodiment, the size of the image of the item to be identified is adjusted by a resizing network.
Specifically, when the width of the object image to be identified is larger than the preset width, scaling the object image to be identified in the width direction to enable the width of the object image to be identified to be equal to the preset width; when the height of the object image to be identified is larger than the preset height, scaling the object image to be identified in the high direction to enable the height of the object image to be identified to be equal to the preset height; when the width of the object image to be identified is smaller than the preset width, carrying out zero padding treatment on the object image to be identified in the width direction, so that the width of the object image to be identified is equal to the preset width; when the height of the image of the object to be identified is smaller than the preset height, carrying out zero padding treatment on the image of the object to be identified in the high direction, so that the height of the image of the object to be identified is equal to the preset height.
In this embodiment, the preset width of the image of the object to be identified is 512 pixels and the preset height is 512 pixels.
Referring to fig. 2, for step S200, an image of an item to be identified is input to a sorting network.
Firstly, the sorting network performs feature extraction according to the object image to be identified to obtain a feature map.
Specifically, the network structure of the feature extraction network adopted for feature extraction according to the image of the object to be identified is Resnet-18 network structure.
Specifically, the Resnet-18 network structure is structured as follows:
The first layer is a convolution layer, convoluting an input image by using a convolution check with the size of 7x7, the step length of 2, the filling of 3 and the channel number of 64, then carrying out batch normalization and ReLU activation operation, and finally carrying out maximum pooling by using a maximum pooling layer with the size of 3x3 and the step length of 2;
The second layer is of a residual structure, the characteristic images output by the first layer are convolved by using convolution cores with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 64, then the characteristic images are convolved in batches, normalized and ReLU activated, the characteristic images output by the first layer are convolved again by using convolution kernels with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 64, the characteristic images are added;
The third layer is of a residual structure, the convolution check of the characteristic images output by the second layer is carried out by using convolution kernels with the size of 3x3, the step length of 1, the filling of 1 and the channel number of 64, then batch normalization and ReLU activation operation are carried out, then the convolution kernels with the size of 3x3, the step length of 1, the filling of 1 and the channel number of 64 are used for convolution again, batch normalization and ReLU activation operation are carried out, and finally the convolution kernels and the characteristic images output by the second layer are added;
The fourth layer is of a residual structure, the characteristic images output by the third layer are convolved by using convolution cores with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 128, then the batch normalization and ReLU activation operation is carried out, the convolution cores with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 128 are convolved again, the batch normalization and ReLU activation operation is carried out, and finally the characteristic images output by the third layer are added with the convolution results of the convolution cores with the size of 1x1, the step size of 2 and the channel number of 128;
The fifth layer is a residual structure, the convolution check of the characteristic images output by the fourth layer is carried out by using the convolution cores with the size of 3x3, the step length of 2, the filling of 1 and the channel number of 128, then the batch normalization and the ReLU activation operation are carried out, the convolution cores with the size of 3x3, the step length of 1, the filling of 1 and the channel number of 128 are used for convolution again, the batch normalization and the ReLU activation operation are carried out, and finally the addition is carried out on the characteristic images output by the fourth layer;
The sixth layer is of a residual structure, the characteristic image output by the fifth layer is subjected to convolution by using a convolution kernel with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 256, then batch normalization and ReLU activation operation are carried out, then the convolution kernel with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 256 is used for convolution again, batch normalization and ReLU activation operation are carried out, and finally the characteristic image output by the fifth layer is added with the result of convolution kernel with the size of 1x1, the step size of 2 and the channel number of 256;
The seventh layer is of a residual structure, the characteristic images output by the sixth layer are convolved by using convolution kernels with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 256, then the characteristic images are convolved by batch normalization and ReLU activation operation, the characteristic images output by the sixth layer are added by using convolution kernels with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 256, and then the characteristic images are convolved again by batch normalization and ReLU activation operation;
The eighth layer is of a residual structure, the characteristic images output by the seventh layer are convolved by using convolution kernels with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 512, then the convolution kernels with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 512 are convolved again, the convolution kernels with the size of 3x3, the step size of 1 and the channel number of 512 are convolved again, the convolution kernels with the size of 1x1, the step size of 2 and the channel number of 512 are convolved, and finally the convolution kernels with the characteristic images output by the seventh layer are added;
The ninth layer is of a residual structure, the characteristic images output by the eighth layer are convolved by using convolution check with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 512, then the batch normalization and the ReLU activation operation are carried out, the convolution kernels with the size of 3x3, the step size of 1, the filling of 1 and the channel number of 512 are used for convolving again, the batch normalization and the ReLU activation operation are carried out, and finally the characteristic images output by the eighth layer are added.
The network is used for cascading a plurality of funnel-shaped networks, so that multi-scale information can be acquired.
After the step of extracting the characteristics according to the image of the object to be identified to obtain the characteristic diagram, the method further comprises the following steps: and enlarging the size of the feature map through a deconvolution network.
The deconvolution network has the following structure:
deconvolution is performed by using a convolution kernel with the size of 4x4, the step size of 2, the filling of 1 and the channel number of 256, batch normalization and ReLU activation operation are performed, deconvolution is performed by using a convolution kernel with the size of 4x4, the step size of 2, the filling of 1 and the channel number of 128, batch normalization and ReLU activation operation are performed, deconvolution is performed by using a convolution kernel with the size of 4x4, the step size of 2, the filling of 1 and the channel number of 64, and batch normalization and ReLU activation operation are performed.
The number of the predicted branches is three, namely, a predicted branch A, a predicted branch B and a predicted branch C, wherein the predicted branch A outputs the central point of a target candidate frame, the predicted branch B outputs the offset value of the target candidate frame, and the predicted branch C outputs the size of the target candidate frame.
The predicted branch a actually outputs a thermodynamic diagram comprising a plurality of keypoints, including the center point of the target candidate box. Predicted branch a contains C channels, each containing a class.
The bias value of the target candidate box output by the prediction branch B can be used for compensating pixel errors caused by mapping the pooled points on the low-temperature map to the original map.
The size of the target candidate box output by the prediction branch C can be used to compensate for the wide and high errors of the target candidate box.
For prediction from feature maps, comprising:
generating a thermodynamic diagram according to the characteristic diagram, and specifically, performing downsampling on the characteristic diagram to generate the thermodynamic diagram;
Scaling the target candidate frame into a thermodynamic diagram, and calculating the center coordinates of a Gaussian circle corresponding to the target candidate frame;
calculating the radius of a Gaussian circle according to the size of the target candidate frame;
calculating a Gaussian value of a Gaussian circle according to the circle center coordinates and the radius;
And taking the position corresponding to the maximum value of the Gaussian values as the center point of the target candidate frame, and outputting the center point of the target candidate frame.
The network structure of the predicted branch is as follows: convolution is performed by using a convolution kernel with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 64, then batch normalization and ReLU activation operations are performed, and finally convolution is performed by using a convolution kernel with the size of 1x1, the step size of 2 and the channel number of 64.
It should be noted that, for some points near the center point of the target candidate frame, when the points are within a certain radius of the center point of the target candidate frame and IoU between the rectangular frame corresponding to the points and the target detection frame is greater than 0.7, the values at the points need to be set to the value of gaussian distribution, that is, the gaussian value, instead of the value 0.
Wherein, the loss function of thermodynamic diagram prediction is:
Alpha and beta are super parameters, the size of alpha is 2, and the size of beta is 4, so that the method is used for balancing difficult and easy samples. Y xyc represents the target value of the target value, The predicted value is represented by N, which is the number of key points, and the key points are points having gaussian values.
When (when)At the time of (1), for the easy-to-classify sample, the predicted value corresponding to the easy-to-classify sampleNear 1, the sample is classifiedA small value is represented, and the value of the loss function is smaller, so that the effect of reducing the weight of the sample is achieved;
When (when) At the time of (1), for the samples which are difficult to classify, the predicted values corresponding to the samples which are difficult to classifyNear 0, the sample is difficult to classifyA large value is indicated, and the value of the loss function is relatively large, which serves to increase the weight of the sample.
When (when)At the time of (1) in order to prevent the predicted valueToo high to be close to 1, usingActing as a penalty term for the loss function. WhileThe closer this parameter is to the center, the smaller its value, and this weight is used to mitigate the penalty.
And predicting according to the feature map, outputting the offset value of the target pre-selected frame and the size of the target candidate frame, wherein the method comprises the following steps:
performing maximum pooling treatment on a plurality of Gaussian values, sequencing the Gaussian values according to the numerical value from large to small, and taking all the Gaussian values ranked before a preset numerical value as target Gaussian values;
And taking the pixel point corresponding to the target pixel value as a target pixel point, and carrying out regression calculation according to the target pixel point to obtain the offset value of the target pre-selected frame and the size of the target candidate frame.
The network structure of the predicted branch corresponding to the bias value of the target pre-selected frame is as follows: convolution is carried out by using a convolution kernel with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 64, then batch normalization and ReLU activation operation are carried out, and finally convolution is carried out by using a convolution kernel with the size of 1x1, the step size of 2 and the channel number of 2.
The network structure of the prediction branch corresponding to the size of the target candidate frame is as follows: convolution is carried out by using a convolution kernel with the size of 3x3, the step size of 2, the filling of 1 and the channel number of 64, then batch normalization and ReLU activation operation are carried out, and finally convolution is carried out by using a convolution kernel with the size of 1x1, the step size of 2 and the channel number of 2.
The loss function of bias value prediction is:
Wherein, Representing the predicted offset value, p representing the image center point coordinates, R representing the scaling factor of the thermodynamic diagram,Representing the approximate certificate coordinates of the scaled center point, the entire process calculates the offset loss of the positive sample block using L1 loss.
The loss function of the size prediction is:
Wherein N represents the number of key points, S k represents the real size of the target, Representing the predicted size, the overall process uses L1L oss to calculate the length-width loss of the positive sample block.
The loss function of the whole prediction network is as follows: l det＝Lk+λsizeLsize+λoffLoff; where λ size and λ off are both weight parameters, where λ size＝0.1,λoff =1.
And then determining a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame and the size of the target candidate frame, and obtaining a recognition result according to the target detection frame.
For step S300, the mechanical arm is controlled to sort objects according to the recognition result.
The training data is input into the sorting network for training, the trained network weight data is migrated to Jetson nano development boards, the sorting network is perfected by using the trained weight data, and the prediction accuracy is improved.
The sorting network is used for identifying the objects, so that the complexity of a network structure and the complexity of network calculation are reduced, the detection performance is improved, the running speed of an algorithm is improved, the detection efficiency is improved, the robustness is good, and the universality is good; thereby improving the sorting efficiency of the objects.
An embodiment of the second aspect of the present invention provides an object sorting apparatus.
Referring to fig. 3, the object sorting apparatus includes an image acquisition unit 10, an image recognition unit 20, and a sorting unit 30.
Wherein the image acquisition unit 10 is used for acquiring an image of an object to be identified; the image recognition unit 20 is configured to input an image of an object to be recognized into a sorting network to perform object recognition to obtain a recognition result, where the sorting network performs feature extraction according to the image of the object to be recognized to obtain a feature map, predicts according to the feature map, outputs a center point of a target candidate frame, a bias value of the target candidate frame, and a size of the target candidate frame through three prediction branches, determines a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame, and the size of the target candidate frame, and obtains the recognition result according to the target detection frame; the sorting unit 30 is used for sorting objects according to the recognition result.
The image acquisition unit 10 may be a device capable of image acquisition with a camera. The image recognition unit 20 may be a computer device with a sorting network. The sorting unit 30 may be a sorting robot apparatus having a mechanical arm.
It should be noted that, each unit of the object sorting device adopted in the embodiment of the second aspect of the present invention corresponds to each step of the object sorting method adopted in the embodiment of the first aspect of the present invention one by one, and both have the same technical scheme, solve the same technical problems, and bring the same technical effects, so the passenger flow prediction device is not described in detail one by one.
An embodiment of a third aspect of the present invention provides an object sorting apparatus. The object sorting apparatus includes: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the object sorting method according to an embodiment of the first aspect of the invention when executing the computer program.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
An embodiment of a fourth aspect of the present invention provides a storage medium. The storage medium stores a computer program for performing the object sorting method according to an embodiment of the first aspect of the invention.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media and communication media. The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM or other memory technology, CD-ROM, digital versatile disks or other optical disk storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The present invention is not limited to the above embodiments, but is merely preferred embodiments of the present invention, and the present invention should be construed as being limited to the above embodiments as long as the technical effects of the present invention are achieved by the same means.
Claims (10)
1. A method of sorting objects, comprising:
Acquiring an image of an object to be identified;
inputting the object image to be identified into a sorting network for object identification to obtain an identification result, wherein the sorting network extracts characteristics according to the object image to be identified to obtain a characteristic diagram, predicts according to the characteristic diagram, outputs a center point of a target candidate frame, a bias value of the target candidate frame and the size of the target candidate frame respectively through three prediction branches, determines a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame and the size of the target candidate frame, and obtains the identification result according to the target detection frame;
and sorting the objects according to the identification result.
2. An object sorting method as claimed in claim 1, characterised in that before the step of inputting the image of the item to be identified into the sorting network, it further comprises:
and adjusting the size of the image of the object to be identified through a size adjustment network.
3. The method of claim 2, wherein said resizing the image of the item to be identified via a resizing network comprises: when the width of the object image to be identified is larger than a preset width, scaling the object image to be identified in the width direction to enable the width of the object image to be identified to be equal to the preset width;
when the height of the object image to be identified is larger than a preset height, scaling the object image to be identified in the high direction to enable the height of the object image to be identified to be equal to the preset height;
when the width of the object image to be identified is smaller than a preset width, carrying out zero padding treatment on the object image to be identified in the width direction so that the width of the object image to be identified is equal to the preset width;
When the height of the object image to be identified is smaller than a preset height, carrying out zero padding on the object image to be identified in the high direction, so that the height of the object image to be identified is equal to the preset height.
4. The object sorting method according to claim 1, wherein predicting based on the feature map, outputting a center point of a target candidate frame, comprises:
generating a thermodynamic diagram according to the characteristic diagram;
Scaling a target candidate frame into the thermodynamic diagram, and calculating center coordinates of a Gaussian circle corresponding to the target candidate frame;
calculating the radius of a Gaussian circle according to the size of the target candidate frame;
calculating a Gaussian value of the Gaussian circle according to the circle center coordinates and the radius;
And taking the position corresponding to the maximum value of the Gaussian value as the center point of the target candidate frame, and outputting the center point of the target candidate frame.
5. The object sorting method according to claim 4, wherein predicting based on the feature map, outputting the bias value of the target candidate frame and the size of the target candidate frame, comprises:
Performing maximum pooling treatment on a plurality of Gaussian values, sequencing the Gaussian values according to the numerical value from large to small, and taking all the Gaussian values ranked before a preset numerical value as target Gaussian values; and taking the pixel point corresponding to the target Gaussian value as a target pixel point, and carrying out regression calculation according to the target pixel point to obtain the bias value of the target candidate frame and the size of the target candidate frame.
6. The object sorting method according to claim 1, further comprising, after the step of extracting features from the image of the object to be identified, the step of:
And enlarging the size of the characteristic map through a deconvolution network.
7. The method according to claim 1, wherein the network structure used for feature extraction from the image of the object to be identified is Resnet-18.
8. An object sorting apparatus, comprising:
The image acquisition unit is used for acquiring an image of the object to be identified;
The image recognition unit is used for inputting the object image to be recognized into a sorting network to perform object recognition to obtain a recognition result, wherein the sorting network performs feature extraction according to the object image to be recognized to obtain a feature image, predicts according to the feature image, respectively outputs a center point of a target candidate frame, a bias value of the target candidate frame and the size of the target candidate frame through three prediction branches, determines a target detection frame according to the center point of the target candidate frame, the bias value of the target candidate frame and the size of the target candidate frame, and obtains the recognition result according to the target detection frame;
And the sorting unit is used for sorting the objects according to the identification result.
9. An object sorting apparatus comprising: memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the object sorting method according to any one of claims 1 to 7 when executing the computer program.
10. A storage medium, characterized in that the storage medium stores a computer program for executing the object sorting method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210460909.XA CN114871115B (en) | 2022-04-28 | 2022-04-28 | Object sorting method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210460909.XA CN114871115B (en) | 2022-04-28 | 2022-04-28 | Object sorting method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114871115A CN114871115A (en) | 2022-08-09 |
| CN114871115B true CN114871115B (en) | 2024-07-05 |
Family
ID=82670925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210460909.XA Active CN114871115B (en) | 2022-04-28 | 2022-04-28 | Object sorting method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114871115B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229455A (en) * | 2017-02-23 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object detecting method, the training method of neural network, device and electronic equipment |
| CN108846415A (en) * | 2018-05-22 | 2018-11-20 | 长沙理工大学 | The Target Identification Unit and method of industrial sorting machine people |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012083855A (en) * | 2010-10-07 | 2012-04-26 | Toyota Motor Corp | Object recognition device and object recognition method |
| CN110210474B (en) * | 2019-04-30 | 2021-06-01 | 北京市商汤科技开发有限公司 | Target detection method and device, equipment and storage medium |
| CN111144322A (en) * | 2019-12-28 | 2020-05-12 | 广东拓斯达科技股份有限公司 | Sorting method, device, equipment and storage medium |
| JP2022522551A (en) * | 2020-02-03 | 2022-04-20 | ベイジン センスタイム テクノロジー ディベロップメント カンパニー リミテッド | Image processing methods and devices, electronic devices and storage media |
| CN113361527B (en) * | 2021-08-09 | 2021-11-19 | 浙江华睿科技股份有限公司 | Multi-target object identification and positioning method and device, electronic equipment and storage medium |
| CN114359624A (en) * | 2021-12-10 | 2022-04-15 | 五邑大学 | Article sorting method and device and storage medium |
-
2022
- 2022-04-28 CN CN202210460909.XA patent/CN114871115B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229455A (en) * | 2017-02-23 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object detecting method, the training method of neural network, device and electronic equipment |
| CN108846415A (en) * | 2018-05-22 | 2018-11-20 | 长沙理工大学 | The Target Identification Unit and method of industrial sorting machine people |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114871115A (en) | 2022-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2858030B1 (en) | Performing a histogram using an array of addressable registers | |
| CN109086811B (en) | Multi-label image classification method and device and electronic equipment | |
| CN108229418B (en) | Human body key point detection method and apparatus, electronic device, storage medium, and program | |
| CN104978738A (en) | Method of detection of points of interest in digital image | |
| CN112435223B (en) | Target detection method, device and storage medium | |
| CN111882565B (en) | Image binarization method, device, equipment and storage medium | |
| CN115345905A (en) | Target object tracking method, device, terminal and storage medium | |
| CN115375917A (en) | Target edge feature extraction method, device, terminal and storage medium | |
| CN109242882B (en) | Visual tracking method, device, medium and equipment | |
| CN114429577A (en) | Flag detection method, system and equipment based on high beacon strategy | |
| CN111178200B (en) | Method for identifying instrument panel indicator lamp and computing equipment | |
| CN116311004B (en) | Video moving target detection method based on sparse optical flow extraction | |
| CN114871115B (en) | Object sorting method, device, equipment and storage medium | |
| CN113569600A (en) | Method and device for identifying weight of object, electronic equipment and storage medium | |
| CN114898306B (en) | Method and device for detecting target orientation and electronic equipment | |
| CN113033256B (en) | Training method and device for fingertip detection model | |
| CN108917768B (en) | Unmanned aerial vehicle positioning navigation method and system | |
| CN111784739A (en) | Target identification method, device, equipment and storage medium | |
| CN111540016A (en) | Pose calculation method and device based on image feature matching, computer equipment and storage medium | |
| CN117523636B (en) | Face detection method and device, electronic equipment and storage medium | |
| Nezhinsky et al. | Efficient and robust shape retrieval from deformable templates | |
| CN113743413B (en) | Visual SLAM method and system combining image semantic information | |
| CN118314315A (en) | Object detection method, device, apparatus, storage medium, and program product | |
| CN116416147A (en) | Image processing method, electronic device, and computer-readable medium | |
| CN113963031A (en) | Depth discrimination type target tracking method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |