CN111428712B - Famous tea picking machine based on artificial intelligence recognition and recognition method for picking machine - Google Patents

Abstract

The invention discloses a famous tea picking machine based on artificial intelligence recognition and a recognition method for a picking machine, which comprises a power system, a control system, a walking system, an illumination system, a vision system, a picking system, a transmission system, an infrared detection system and a collection box, wherein the power system is used for driving the picking system to pick tea; the power system drives the traveling system to travel forwards under the action of the control system, four two-eye cameras are used as a visual system of the famous tea picking harvester, after image acquisition is completed, the images are uploaded to a control system server through a local or 5G cloud, tea leaves are identified and positioned by using an image identification technology and an artificial intelligence method, the picking system is moved and rotated to a proper position through a spider hand robot in the picking system, a tail end picker of the picking system is used for breaking and clamping tea young tips, then the tea young tips are conveyed to the conveying system and are conveyed to the collecting box through the conveying system, and the collecting box is stored in a layered mode.

Description

Famous tea picking machine based on artificial intelligence recognition and recognition method for picking machine

Technical Field

The invention relates to the technical field of artificial intelligence recognition and mechanical tea picking, in particular to a famous tea picking machine based on artificial intelligence recognition and a recognition method for a picking machine.

Background

Tea picking is labor-intensive, seasonal and short in picking period, and famous tea is more strict in picking time and standard. The picking of tea not only relates to the quality, yield and economic benefit of tea, but also has influence on the nutrient content of tea and the later growth of tea trees.

The tea leaf picking mode is mainly manual picking and mechanical picking.

The tea picking method has the advantages that the manual picking is selective, the tea quality is high, people can judge whether the tea buds are suitable for picking and the grade of the tea buds by observing the characteristics of the shapes, the colors and the like of the tea buds, and pick the tea tender buds in a breaking mode, so that the completeness of the bud shapes is ensured, the tea tender shoots have higher quality, the cost is high, and the problem of serious shortage of labor force can be caused in tea picking seasons every year due to the optimized adjustment of industrial structures and the transfer of labor force.

The existing mechanical tea plucker mainly comprises a single-person tea plucker, a double-person lifting tea plucker, a self-propelled tea plucker and the like. The mechanical picking mechanism is a reciprocating cutting type, namely a working mode of 'one-knife cutting', and the tea plucking machine has low cost and high efficiency. But missed picking and wrong picking can be generated, the integrity of the leaves can not be effectively ensured, the picking mode has no selectivity, and various requirements of famous tea can not be met.

Disclosure of Invention

The invention provides an artificial intelligence-based famous tea picking machine, which solves the problems of missed picking and mistaken picking in the existing mechanical tea picking machine, incapability of effectively ensuring the integrity of leaves, no selectivity in picking and incapability of meeting various requirements of famous tea aiming at the defects of manual picking and the existing mechanical tea picking machine.

The technical scheme of the invention is as follows: a famous tea picking machine based on artificial intelligence recognition comprises a power system, a control system, a walking system, an illumination system, a vision system, a picking system, a transmission system, an infrared detection system and a collecting box; the power system drives the traveling system to travel forwards under the action of the control system, four two-eye cameras are used as a visual system of the famous tea picking harvester, after image acquisition is completed, the images are uploaded to a control system server through a local or 5G cloud, then the tea leaves are identified and positioned by using an image identification technology and an artificial intelligence method, the picking system is moved and rotated to a proper position through a spider hand robot in the picking system, a tail end picker of the picking system is used for breaking and clamping the fresh tips of the tea leaves, then the tea leaves are conveyed to the conveying system and are conveyed to the collecting box through the conveying system, and the collecting box is stored in a layered mode; the illumination system provides a stable illumination environment for the vision system; and the infrared detection system is used for detecting whether the famous tea picking harvester exceeds the working range or not, sending out a corresponding signal in time and uploading the signal to the control system, and the control system immediately sends out a signal for continuing operation or stopping.

In the above, the control system is respectively connected with the walking system, the illumination system, the picking system, the infrared detection system and the transmission system; the control system is connected with the walking system and used for reasonably planning the motion trail of the picking harvester; the control system is connected with the illumination system and used for controlling the light of the illumination system to be switched on and off; the control system is connected with the picking system and used for carrying out real-time control management on the spider hand robot and the tail end picker, and regulating and controlling the position and speed parameters of the moving part to enable the mechanical device to operate according to an expected track or specified actions; the control system is connected with the infrared detection system and used for receiving signals sent by the infrared detection system and making corresponding control; the control system is connected with the transmission system and controls whether the transmission system operates or not.

In the above, the vision system adopts four binocular cameras as the vision system of the famous tea picking harvester, and the cameras are positioned below the static platform; the vision system identifies the tea leaves by an image identification technology and an artificial intelligence method and provides accurate coordinate positions of the tea leaves for the control system, and the control system controls the picking system to pick the tea leaves accurately.

In the above, the infrared detection system is positioned at the upper part in front of the machine and is used for detecting whether the famous tea picking harvester exceeds the working range, sending out corresponding signals in time and uploading the signals to the control system; the collecting box is positioned at the rear part of the famous high-quality tea picking harvester and is used for collecting and storing tea tender shoots transmitted from the transmission system.

In the above, the picking system is divided into a spider mobile robot and a tail end picker; the spider hand robot comprises a static platform, a dynamic platform, a driving motor, a driving arm and a driven arm; the tail end of the static platform is connected with the driving arm and the driven arm through hinges respectively to form a telescopic and deformable triangle, and the lower end of the movable platform is provided with a tail end picker; the mode of human finger picking is simulated, the two picking arms have a certain height difference, the first picking arm and the second picking arm move in a staggered mode during working, the bud shape integrity of tea tender shoots is guaranteed through breaking operation of specific picking points of the tea tender shoots, and the tea tender shoots are clamped through the clamping arms.

In the above, the visual system identifies the tea tender shoots through an image identification technology and artificial intelligence, and specifically comprises the following steps:

step 1: shooting tea images of a tea garden, preprocessing the tea images, firstly, selecting clear and definite images of tea, secondly, dividing the selected training set images into a training set and a testing set according to the proportion of 7:3, marking the training set images by using Labelimg software, and storing the training set images in an XML file format;

step 2: after image preprocessing, inputting the marked original images of the tea leaves and the corresponding XML files into a built Faster R-CNN model one by one for training, adjusting various parameters to enable the recognition effect of the model to be optimal, storing the trained model and uploading the model to the cloud for use when a machine picks the model;

and step 3: acquiring target identification, class score, coordinates and time parameters through a trained Faster R-CNN model, and storing the data;

and 4, step 4: after the stored data are preprocessed, the identification and the positioning of the tea leaves in the tea garden are realized.

In the above, in the step 2, the established Faster R-CNN model specifically includes the following steps:

step (1): the VGG16 network convolution is used to extract the image features of tea: the input is an image of an arbitrary size, and the output is a feature map including extracted features. The VGG16 network convolution consists of 13 convolution layers, 13 ReLU function layers and 4 pooling layers; the specific situation is as follows: the convolutional layer: the method is used for realizing parameter sharing; the size of the image is not changed at the convolutional layer, i.e. the size of the input image O is equal to the size of the output image; the output of the convolutional layer is formula 1:

equation 1:

wherein O is an output image, I is an input image, K is the size of a convolution kernel, P is the size of an amplification matrix, and S is a step length;

the ReLU layer: ReLU is an activation function that does not change the size of the image; the introduced nonlinearity of the ReLU function is used, so that the output of part of neurons is zero, the interdependence among parameters is reduced, and the occurrence of an overfitting problem is relieved; the pooling layer: for reducing the spatial size of the image, reducing the size of the output image to half the size of the input image;

step (2): after the tea image is subjected to network convolution by VGG16, an image different from an original image, namely a characteristic map, is obtained, and the characteristic map distinguishes the color of a target part from the background so as to highlight the difference of the target;

and (3): the region suggestion network RPN is used for generating region suggestions and transmitting the obtained suggestions to the region of interest for pooling; the specific steps for realizing the RPN of the regional proposal network are as follows:

step A: convolving the feature map output by the VGG16 network by using a 3 × 3 sliding window, and reducing the dimension by using two 1 × 1 sliding windows to reduce the calculation parameters and accelerate the operation speed; wherein, a sliding window of 1 × 1 will give suggestions directly, and another sliding window of 1 × 1 will proceed with the following steps to give suggestions;

and B: after convolution through a 1 × 1 sliding window, parameter adjustment is performed, and the offset between an anchor point frame and a true value is evaluated to make a more accurate suggestion, where the offset is calculated as formula 2:

equation 2:

wherein x and y represent the coordinates of the center point of the true value; w and h represent the width and height, respectively, of the true value; x is the number of_aAnd y_aRepresenting the coordinates of the center point of the anchor point frame; w is a_aAnd h_aRespectively representing the width and height of the anchor frame; delta x and delta y are real values and offset of coordinates of the center point of the anchor point frame; Δ w and Δ h represent the width and height offsets between the true value and the anchor box, respectively;

and C: the softmax classifier screens the offset made in the previous step, and reasonably keeps a plurality of numerical values according to the set threshold parameter, so that a region suggestion of the target frame is given;

step D: drawing a plurality of bounding boxes of the target corresponding to a plurality of numerical values given by the softmax classifier, simultaneously reserving the bounding boxes of the target, and transmitting the bounding boxes of the target to the next step;

step E: according to a non-maximum inhibition method, selecting a plurality of target frames generated in the previous step, and reserving one which can cover the target and has a higher classification score as a suggestion;

step F: and integrating the suggestions respectively given by the two 1 × 1 sliding windows to obtain a total suggestion, and transmitting the total suggestion to the region of interest for pooling.

Step (4), pooling of the region of interest: and meanwhile, a final target position frame is obtained through the prediction of a boundary frame, and a final target prediction score is obtained after class scores are screened through a softmax classifier, so that an image with a target mark frame and a prediction score is obtained.

The invention also provides an identification method for the famous and high-quality tea picker based on artificial intelligence identification, which comprises the following steps of:

step 1: shooting tea images of a tea garden, preprocessing the tea images, firstly, selecting clear and definite images of tea, secondly, dividing the selected training set images into a training set and a testing set according to the proportion of 7:3, marking the training set images by using Labelimg software, and storing the training set images in an XML file format;

step 2: after image preprocessing, inputting the marked original images of the tea leaves and the corresponding XML files into a built Faster R-CNN model one by one for training, adjusting various parameters to enable the recognition effect of the model to be optimal, storing the trained model and uploading the model to the cloud for use when a machine picks the model;

and step 3: acquiring target identification, class score, coordinates and time parameters through a trained Faster R-CNN model, and storing the data;

and 4, step 4: after the stored data are preprocessed, the identification and the positioning of the tea leaves in the tea garden are realized.

In the step 2, the established Faster R-CNN model specifically comprises the following steps:

step (1): the VGG16 network convolution is used to extract the image features of tea: the input is an image of an arbitrary size, and the output is a feature map including extracted features. The VGG16 network convolution consists of 13 convolution layers, 13 ReLU function layers and 4 pooling layers; the specific situation is as follows: the convolutional layer: the method is used for realizing parameter sharing; the size of the image is not changed at the convolutional layer, i.e. the size of the input image O is equal to the size of the output image; the output of the convolutional layer is formula 1:

equation 1:

wherein O is an output image, I is an input image, K is the size of a convolution kernel, P is the size of an amplification matrix, and S is a step length;

the ReLU layer: ReLU is an activation function that does not change the size of the image; the introduced nonlinearity of the ReLU function is used, so that the output of part of neurons is zero, the interdependence among parameters is reduced, and the occurrence of an overfitting problem is relieved; the pooling layer: for reducing the spatial size of the image, reducing the size of the output image to half the size of the input image;

step (2): after the tea image is subjected to network convolution by VGG16, an image different from an original image, namely a characteristic map, is obtained, and the characteristic map distinguishes the color of a target part from the background so as to highlight the difference of the target;

and (3): the region suggestion network RPN is used for generating region suggestions and transmitting the obtained suggestions to the region of interest for pooling;

step (4), pooling of the region of interest: and meanwhile, a final target position frame is obtained through the prediction of a boundary frame, and a final target prediction score is obtained after class scores are screened through a softmax classifier, so that an image with a target mark frame and a prediction score is obtained.

In the step (3), the specific steps for implementing the regional proposal network RPN are as follows:

step A: convolving the feature map output by the VGG16 network by using a 3 × 3 sliding window, and reducing the dimension by using two 1 × 1 sliding windows to reduce the calculation parameters and accelerate the operation speed; wherein, a sliding window of 1 × 1 will give suggestions directly, and another sliding window of 1 × 1 will proceed with the following steps to give suggestions;

and B: after convolution through a 1 × 1 sliding window, parameter adjustment is performed, and the offset between an anchor point frame and a true value is evaluated to make a more accurate suggestion, where the offset is calculated as formula 2:

equation 2:

wherein x and y represent the coordinates of the center point of the true value; w and h represent the width and height, respectively, of the true value; x is the number of_aAnd y_aRepresenting the coordinates of the center point of the anchor point frame; w is a_aAnd h_aRespectively representing the width and height of the anchor frame; delta x and delta y are real values and offset of coordinates of the center point of the anchor point frame; Δ w and Δ h represent the width and height offsets between the true value and the anchor box, respectively;

and C: the softmax classifier screens the offset made in the previous step, and reasonably keeps a plurality of numerical values according to the set threshold parameter, so that a region suggestion of the target frame is given;

step D: drawing a plurality of bounding boxes of the target corresponding to a plurality of numerical values given by the softmax classifier, simultaneously reserving the bounding boxes of the target, and transmitting the bounding boxes of the target to the next step;

step E: according to a non-maximum inhibition method, selecting a plurality of target frames generated in the previous step, and reserving one which can cover the target and has a higher classification score as a suggestion;

step F: and integrating the suggestions respectively given by the two 1 × 1 sliding windows to obtain a total suggestion, and transmitting the total suggestion to the region of interest for pooling.

The famous tea picking harvester based on artificial intelligence combines the artificial intelligence with a spider mobile robot, and adopts a bionic design hand picking method to pick tender tea shoots in a breaking mode, so that famous tea can be picked quickly and efficiently. Famous tea picking harvester based on artificial intelligence has the advantages that: 1. the picking is selective, tender tea shoots are picked in a breaking mode, the old and tender tea shoots are strictly distinguished, the mistaken picking rate, the missing picking rate and the breakage rate are reduced, and various index requirements of famous tea are met. 2. During the working process of the famous tea picking harvester, tea trees and tea tender shoots are not damaged. 3. The famous tea picking harvester can work continuously for 24 hours in all weather without being influenced by factors such as time, weather and the like. 4. The famous tea picking harvester can simultaneously carry out multi-region cooperative work, has high working efficiency and greatly relieves the problem of insufficient labor. 5. The famous tea picking harvester has an autonomous operation function, and can automatically stop operation when reaching the tail end of a tea ridge and send a corresponding signal to a control system. 6. The tea leaf recognition and positioning based on the deep learning algorithm greatly improves the picking precision of the picking system. 7. Based on the tea tender shoot and young sprout cloud decision-making system for 5G transmission, the picking efficiency is greatly improved.

Drawings

Fig. 1 is a schematic view of a spider hand robot in the picking system of the present invention.

Fig. 2 is a schematic view of an end picker in the picking system of the present invention.

Fig. 3 is a schematic diagram of the working state of the transmission system of the present invention.

FIG. 4 is a schematic diagram of the framework of the present invention based on Faster R-CNN.

Fig. 5 is a schematic view of the overall structure of the picking machine of the present invention.

Detailed Description

In order to facilitate an understanding of the invention, the invention is described in more detail below with reference to the accompanying drawings and specific examples. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

Example 1

One embodiment of the present invention is, as shown in fig. 5: a famous tea picking machine based on artificial intelligence comprises a power system 1, a control system, a walking system 3, an illumination system 4, a vision system 5, a picking system 6, a transmission system, an infrared detection system 8 and a collection box 9; the tea leaf picking machine is characterized in that the power system 1 drives the traveling system 3 to move forwards under the action of the control system, four two-eye cameras are used as a visual system 5 of the famous tea picking harvester, after image acquisition is completed, the images are uploaded to a control system server through a local or 5G cloud, tea leaves are identified and positioned by using an image identification technology and an artificial intelligence method, the picking system is moved and rotated to a proper position through a spider hand robot in the picking system 6, a tea leaf young tip is broken and clamped by using a tail end picker of the picking system 6 and then conveyed to a conveying system, the tea leaves are conveyed to a collecting box 9 by the conveying system, and the collecting box 9 is stored in a layered mode. Wherein, the illumination system 4 provides a stable illumination environment for the vision system; and the infrared detection system 8 is used for detecting whether the famous tea picking harvester exceeds the working range or not, sending out a corresponding signal in time and uploading the signal to the control system, and the control system immediately sends out a signal for continuing operation or stopping.

A power system 1: an electric power system can be used as an energy system of the harvester, so that the influence of complex structure, vibration, environmental pollution and the like caused by the adoption of a fuel oil machine is avoided. Non-electric power systems may also be employed.

The control system specifically comprises: 1. and the walking system is connected with the picking harvester and is used for reasonably planning the motion track of the picking harvester. 2. And the switch is connected with the illumination system and controls the light of the illumination system. 3. And the robot is connected with a picking system, controls and manages the spider hand robot and the tail end picker in real time, and regulates and controls parameters such as the position, the speed and the like of a moving part, so that the mechanical device operates according to an expected track or specified action. 4. And the infrared detection system is connected with the infrared detection system, receives the signal sent by the infrared detection system and performs corresponding control. 5. And the transmission system is connected with the transmission system to control whether the transmission system operates or not.

A traveling system 3: the electric crawler is adopted as the traveling mechanism of the famous tea picking harvester, the crawler has certain cross-country performance, the requirements of different soil qualities and terrains can be met, the protective capability is stronger compared with a wheel type mechanism, the gravity center of a chassis of the famous tea picking harvester is lower, and the stability of the structure is stronger. The traveling system is connected with the control system through the power system, the distance of the famous tea picking harvester in each traveling is 1 meter, then the famous tea picking harvester stops to perform image acquisition and tea picking, and the traveling operation is performed again after the picking is completed.

The illumination system 4: the vision system of the famous tea picking harvester needs a stable illumination environment, the top and the periphery of the vision system are sealed by plates and a frame is built by adopting aluminum profiles, the top plate is also used for mounting a robot static platform, certain strength and rigidity are ensured, and the side plate sealing is mainly used for isolating an external light source. In addition, an independent light source is needed inside the tea-picking harvester, so light source mounting interfaces are arranged on the inner sides of the front plate, the rear plate and the side plates.

The visual system 5: four binocular cameras are used as a visual system of the famous high-quality tea picking harvester, and the cameras are located below the static platform. The vision system recognizes the tea shoots by image recognition techniques and provides the control system with their exact coordinate positions. The method is based on 5G ultra-high-speed image uploading, and remote cloud decision is realized; the method is characterized in that target detection and positioning of tender shoot young shoots are achieved based on deep learning, a depth camera imaging technology, an image processing technology, a cloud computing technology, a 5G communication transmission technology and an embedded system development technology are integrated into a project, and the project is applied to tea identification and positioning. After information such as position coordinates of tea leaves is obtained, accurate picking of the tea leaves is carried out by combining the machine spider hand robot, so that the tea leaf picking and harvesting machine is intelligent.

And (6) a picking system: when the famous tea picking harvester works, the picking area is automatically divided into A, B, C, D four areas, the four areas can be operated simultaneously, and each area does not interfere, so that the operation efficiency is greatly improved. The picking system comprises a spider mobile phone robot and a tail end picker. The spider hand robot can realize the accurate location of machine fast. As shown in fig. 1, the spider-hand robot includes a stationary platform 101, a movable platform 105, a drive motor 103, a master arm 102, and a slave arm 104. The spider-hand robot can stretch to different lengths, the tail end of the static platform 101 is connected with the driving arm 102 and the driven arm 104 through hinges respectively to form a telescopic and deformable triangle, and the tail end picker is arranged at the lower end of the movable platform 105 and can be driven to change the left position, the right position and the height according to the height of tea tender shoots.

The end picking device is designed from the perspective of bionics, the mode of human finger picking is simulated, the two picking arms have a certain height difference, and when the end picking device works, as shown in fig. 2, the first picking arm 201 and the second picking arm 202 move in a staggered mode, the bud shape integrity of tea tender shoots is guaranteed through breaking operation of specific picking points of the tea tender shoots, and the tea tender shoots are clamped through the clamping arms 203. The picking system can be provided with a plurality of groups according to the width of the tea ridge and can also pick in grades according to the needs. The end picker is driven by a motor and can rotate and break and clamp the tender shoots of the tea leaves.

As shown in fig. 3, the transmission system adopts a semi-sealed L-shaped transmission mechanism, and is not interfered by external factors. When the picking machine needs to run, the transmission system stops transmission work of the newly picked tea tender shoots in the running process, and the L-shaped transmission mechanism is disconnected as shown in the state I in figure 3, so that the picking machine can run conveniently. When newly-picked tea tender shoots need to be rapidly and stably transmitted to the collection box, the L-shaped transmission mechanism of the transmission system is in a parallel state, as shown in a state II in figure 3, and after the transmission system is connected, the newly-picked tea tender shoots are accurately and rapidly and stably transmitted to the collection box.

The infrared detection system 8: the tea picking and harvesting machine is positioned at the upper part in front of the machine and used for detecting whether the famous and high-quality tea picking and harvesting machine exceeds a working range or not, sending a corresponding signal in time and uploading the signal to the control system, and the control system immediately sends a signal for continuing working or stopping.

The collection box 9: and the tea leaf picking device is positioned at the rear part of the famous tea picking harvester and is used for collecting and storing tea leaf buds transmitted from the transmission system. The collecting box is provided with a layering storage mechanism, when the fresh tea tender shoots of each layer reach a certain number, the stress of the spring reaches a certain limit, the layering mechanism is automatically opened, the tea leaves fall to the next layer, and finally the tea leaves fall layer by layer. Thereby reducing damage to the fresh and tender tea shoots caused by the squeezing.

The invention utilizes an image recognition technology and an artificial intelligence method to recognize and position tea leaves, and specifically comprises the following steps:

step 1: shooting tea images of a tea garden, preprocessing the tea images, firstly, selecting clear and definite images of tea, secondly, dividing the selected training set images into a training set and a testing set according to the proportion of 7:3, marking the training set images by using Labelimg (Windows environment) software, and storing the training set images in an XML file format;

step 2: after image preprocessing, inputting the marked original images of the tea leaves and the corresponding XML files into a built Faster R-CNN model one by one for training, adjusting various parameters to enable the recognition effect of the model to be optimal, storing the trained model and uploading the model to the cloud for use when a machine picks the model;

and step 3: and acquiring target identification, class score, coordinate and time parameter through the trained Faster R-CNN model, and storing the data.

And 4, step 4: after the stored data are preprocessed, the identification and the positioning of the tea leaves in the tea garden are realized.

In the step 2, the fast R-CNN model is established based on the tea recognition and positioning of a Deep Learning algorithm, and Deep Learning (Deep Learning) is one of the techniques which are rapidly developed in the artificial intelligence field in recent two years and is widely applied to the fields of voice recognition, text translation, image recognition and the like. As one of the algorithms for deep learning, yolo (young Only Look once) is a single neural network-based target detection system proposed by Joseph Redmon and Ali faradai et al in 2015. In 2017, the CVPR congress published YOLO V2, and the detection precision and speed are further improved. In one year, the YOLO V3 algorithm published by the people greatly improves the operation recognition speed on the basis of further improving the recognition accuracy rate, can finish faster video tracking recognition, and is widely applied to the field of automatic driving. As another algorithm for deep learning, Fast R-CNN is the development of Shaoqing Ren et al on the basis that R-CNN can not realize end-to-end training and Fast R-CNN selective search consumes time, has the advantages of Fast training model and Fast recognition speed, and has higher accuracy in recognizing objects, and is one of the algorithms which are considered to be used preferentially. The appearance of algorithms such as YOLO and Faster R-CNN provides possibility for a rapid video image target detection technology, and provides a theoretical basis for picking, identifying and positioning the tea in the patent.

Based on the existing work, the invention takes the Faster R-CNN model algorithm as an example to introduce the specific identification process. The method comprises the steps of shooting tea images of a tea garden in advance, using the tea images as training set images, inputting the training set images into a built Faster R-CNN model after being marked by Labelimg software, training, enabling the recognition effect of the model to be optimal by adjusting various parameters, storing the trained model and uploading the model to the cloud for use when a machine picks the tea. The internal structure of the Faster R-CNN model is shown in FIG. 4, and is specifically as follows:

step (1): the VGG16 network convolution is used to extract the image features of tea: the input is an image of an arbitrary size, and the output is a feature map including extracted features. The VGG16 network convolution consists of 13 convolution layers, 13 ReLU function layers and 4 pooling layers. The specific situation is as follows:

1) and (3) rolling layers: the function is to implement parameter sharing. This layer does not change the size of the image, i.e. the size of the input image O is equal to the size of the output image. The output of this layer is equation 1:

equation 1:

where O is the output image, I is the input image, K is the size of the convolution kernel, P is the size of the augmented matrix, and S is the step size.

2) ReLU layer: ReLU is an activation function that does not change the size of the image. The ReLU function is used for introducing nonlinearity, so that the output of partial neurons is zero, the interdependence among parameters is reduced, and the occurrence of an overfitting problem is relieved.

3) A pooling layer: its effect is to reduce the spatial size of the image, reducing the size of the output image to half the size of the input image.

Step (2): after the tea image is subjected to network convolution by VGG16, an image different from the original image, namely a feature map, is obtained, and the color of the target part is distinguished from the background by the feature map so as to highlight the difference of the target.

And (3): the region proposal network RPN is used to generate region proposals and to transmit the derived proposals to the region of interest pooling. The specific steps for realizing the RPN of the regional proposal network are as follows:

step A: convolving the feature map output by the VGG16 network by using a 3 × 3 sliding window, and reducing the dimension by using two 1 × 1 sliding windows to reduce the calculation parameters and accelerate the operation speed. One sliding window of 1 × 1 will give suggestions directly, and the other sliding window of 1 × 1 will go through the following steps to give suggestions;

and B: after convolution through a 1 × 1 sliding window, parameter adjustment is performed, and the offset between an anchor point frame and a true value is evaluated to make a more accurate suggestion, where the offset is calculated as formula 2:

equation 2:

wherein x and y represent the coordinates of the center point of the true value; w and h represent the width and height, respectively, of the true value. x is the number of_aAnd y_aRepresenting the coordinates of the center point of the anchor point frame; w is a_aAnd h_aRespectively representing the width and height of the anchor frame; delta x and delta y are real values and offset of coordinates of the center point of the anchor point frame; Δ w and Δ h represent the wide and high offsets between the true values and the anchor boxes, respectively.

And C: the softmax classifier screens the offset made in the previous step, and reasonably keeps a plurality of numerical values according to the set threshold parameter, so that a region suggestion of the target frame is given;

step D: drawing a plurality of bounding boxes of the target corresponding to a plurality of numerical values given by the softmax classifier, simultaneously reserving the bounding boxes of the target, and transmitting the bounding boxes of the target to the next step;

step E: according to a non-maximum inhibition method, selecting a plurality of target frames generated in the previous step, and reserving one which can cover the target and has a higher classification score as a suggestion;

step F: and integrating the suggestions respectively given by the two 1 × 1 sliding windows to obtain a total suggestion, and transmitting the total suggestion to the region of interest for pooling.

And (4): region of interest pooling: and meanwhile, a final target position frame is obtained through the prediction of a boundary frame, and a final target prediction score is obtained after class scores are screened through a softmax classifier, so that an image with a target mark frame and a prediction score is obtained.

Tea tender shoot young sprout cloud decision-making system based on 5G transmission: the 5G greatly improves the transmission speed, and makes the use of cloud storage and decision possible. Therefore, this patent uses 5G transmission and high in the clouds storage module, comprises 5G transmission module and high in the clouds memory. The 5G transmission module is connected with the camera module, images shot by the camera on line can be uploaded to the cloud storage at a high speed through the 5G transmission module, the cloud storage can store corresponding deep learning models in advance, tea leaves in the images are positioned and identified after the images are received, and the obtained tea leaf position information is transmitted to the cloud decision-making system. Signals decided by the cloud decision system can also be transmitted to the three-dimensional spider hand control module at a high speed through the 5G transmission module, so that the spider hand is controlled to move, and therefore, the fresh tea tender shoots are picked through the tail end picker. With the help of high in the clouds decision-making system can save the process of using computer or manual operation on machinery, great improvement the efficiency of picking for machinery really realizes automaticly, possesses artificial intelligence's characteristics.

The famous tea picking harvester based on artificial intelligence combines the artificial intelligence with a spider mobile robot, and adopts a bionic design hand picking method to pick tender tea shoots in a breaking mode, so that famous tea can be picked quickly and efficiently. Famous tea picking harvester based on artificial intelligence has the advantages that: 1. the picking is selective, tender tea shoots are picked in a breaking mode, the old and tender tea shoots are strictly distinguished, the mistaken picking rate, the missing picking rate and the breakage rate are reduced, and various index requirements of famous tea are met. 2. During the working process of the famous tea picking harvester, tea trees and tea tender shoots are not damaged. 3. The famous tea picking harvester can work continuously for 24 hours in all weather without being influenced by factors such as time, weather and the like. 4. The famous tea picking harvester can simultaneously carry out multi-region cooperative work, has high working efficiency and greatly relieves the problem of insufficient labor. 5. The famous tea picking harvester has an autonomous operation function, and can automatically stop operation when reaching the tail end of a tea ridge and send a corresponding signal to a control system. 6. The tea leaf recognition and positioning based on the deep learning algorithm greatly improves the picking precision of the picking system. 7. Based on the tea tender shoot and young sprout cloud decision-making system for 5G transmission, the picking efficiency is greatly improved.

Example 2:

on the basis of embodiment 1, the invention provides an identification method for a famous tea picker based on artificial intelligence identification, which specifically comprises the following steps:

step 1: shooting tea images of a tea garden, preprocessing the tea images, firstly, selecting clear and definite images of tea, secondly, dividing the selected training set images into a training set and a testing set according to the proportion of 7:3, marking the training set images by using Labelimg (Windows environment) software, and storing the training set images in an XML file format;

step 2: after image preprocessing, inputting the marked original images of the tea leaves and the corresponding XML files into a built Faster R-CNN model one by one for training, adjusting various parameters to enable the recognition effect of the model to be optimal, storing the trained model and uploading the model to the cloud for use when a machine picks the model;

and step 3: and acquiring target identification, class score, coordinate and time parameter through the trained Faster R-CNN model, and storing the data.

And 4, step 4: after the stored data are preprocessed, the identification and the positioning of the tea leaves in the tea garden are realized.

In step 2, a fast R-CNN model is established, as shown in fig. 4, based on the Deep Learning algorithm, the tea leaf recognition and positioning is performed, and Deep Learning (Deep Learning) is one of the techniques that are rapidly developed in the artificial intelligence field in recent two years, and is widely applied to the fields of speech recognition, text translation, image recognition, and the like. As one of the algorithms for deep learning, yolo (young Only Look once) is a single neural network-based target detection system proposed by Joseph Redmon and Ali faradai et al in 2015. In 2017, the CVPR congress published YOLO V2, and the detection precision and speed are further improved. In one year, the YOLO V3 algorithm published by the people greatly improves the operation recognition speed on the basis of further improving the recognition accuracy rate, can finish faster video tracking recognition, and is widely applied to the field of automatic driving. As another algorithm for deep learning, Fast R-CNN is the development of Shaoqing Ren et al on the basis that R-CNN can not realize end-to-end training and Fast R-CNN selective search consumes time, has the advantages of Fast training model and Fast recognition speed, and has higher accuracy in recognizing objects, and is one of the algorithms which are considered to be used preferentially. The appearance of algorithms such as YOLO and Faster R-CNN provides possibility for a rapid video image target detection technology, and provides a theoretical basis for picking, identifying and positioning the tea in the patent.

Based on the existing work, the invention takes the Faster R-CNN model algorithm as an example to introduce the specific identification process. The method comprises the steps of shooting tea images of a tea garden in advance, using the tea images as training set images, inputting the training set images into a built Faster R-CNN model after being marked by Labelimg software, training, enabling the recognition effect of the model to be optimal by adjusting various parameters, storing the trained model and uploading the model to the cloud for use when a machine picks the tea. The internal structure of the Faster R-CNN model is as follows:

step (1): the VGG16 network convolution is used to extract the image features of tea: the input is an image of an arbitrary size, and the output is a feature map including extracted features. The VGG16 network convolution consists of 13 convolution layers, 13 ReLU function layers and 4 pooling layers. The specific situation is as follows:

1) and (3) rolling layers: the function is to implement parameter sharing. This layer does not change the size of the image, i.e. the size of the input image O is equal to the size of the output image. The output of this layer is equation 1:

equation 1:

where O is the output image, I is the input image, K is the size of the convolution kernel, P is the size of the augmented matrix, and S is the step size.

2) ReLU layer: ReLU is an activation function that does not change the size of the image. The ReLU function is used for introducing nonlinearity, so that the output of partial neurons is zero, the interdependence among parameters is reduced, and the occurrence of an overfitting problem is relieved.

3) A pooling layer: its effect is to reduce the spatial size of the image, reducing the size of the output image to half the size of the input image.

Step (2): after the tea image is subjected to network convolution by VGG16, an image different from the original image, namely a feature map, is obtained, and the color of the target part is distinguished from the background by the feature map so as to highlight the difference of the target.

And (3): the region proposal network RPN is used to generate region proposals and to transmit the derived proposals to the region of interest pooling. The specific steps for realizing the RPN of the regional proposal network are as follows:

step A: convolving the feature map output by the VGG16 network by using a 3 × 3 sliding window, and reducing the dimension by using two 1 × 1 sliding windows to reduce the calculation parameters and accelerate the operation speed. One sliding window of 1 × 1 will give suggestions directly, and the other sliding window of 1 × 1 will go through the following steps to give suggestions;

and B: after convolution through a 1 × 1 sliding window, parameter adjustment is performed, and the offset between an anchor point frame and a true value is evaluated to make a more accurate suggestion, where the offset is calculated as formula 2:

equation 2:

wherein x and y represent the coordinates of the center point of the true value; w and h represent the width and height, respectively, of the true value. x is the number of_aAnd y_aRepresenting the coordinates of the center point of the anchor point frame; w is a_aAnd h_aRespectively representing the width and height of the anchor frame; delta x and delta y are real values and offset of coordinates of the center point of the anchor point frame; Δ w and Δ h represent the wide and high offsets between the true values and the anchor boxes, respectively.

And C: the softmax classifier screens the offset made in the previous step, and reasonably keeps a plurality of numerical values according to the set threshold parameter, so that a region suggestion of the target frame is given;

step D: drawing a plurality of bounding boxes of the target corresponding to a plurality of numerical values given by the softmax classifier, simultaneously reserving the bounding boxes of the target, and transmitting the bounding boxes of the target to the next step;

step E: according to a non-maximum inhibition method, selecting a plurality of target frames generated in the previous step, and reserving one which can cover the target and has a higher classification score as a suggestion;

step F: and integrating the suggestions respectively given by the two 1 × 1 sliding windows to obtain a total suggestion, and transmitting the total suggestion to the region of interest for pooling.

And (4): region of interest pooling: and meanwhile, a final target position frame is obtained through the prediction of a boundary frame, and a final target prediction score is obtained after class scores are screened through a softmax classifier, so that an image with a target mark frame and a prediction score is obtained.

Tea tender shoot young sprout cloud decision-making system based on 5G transmission: the 5G greatly improves the transmission speed, and makes the use of cloud storage and decision possible. Therefore, this patent uses 5G transmission and high in the clouds storage module, comprises 5G transmission module and high in the clouds memory. The 5G transmission module is connected with the camera module, images shot by the camera on line can be uploaded to the cloud storage at a high speed through the 5G transmission module, the cloud storage can store corresponding deep learning models in advance, tea leaves in the images are positioned and identified after the images are received, and the obtained tea leaf position information is transmitted to the cloud decision-making system. Signals decided by the cloud decision system can also be transmitted to the three-dimensional spider hand control module at a high speed through the 5G transmission module, so that the spider hand is controlled to move, and therefore, the fresh tea tender shoots are picked through the tail end picker. With the help of high in the clouds decision-making system can save the process of using computer or manual operation on machinery, great improvement the efficiency of picking for machinery really realizes automaticly, possesses artificial intelligence's characteristics.

The famous tea picking harvester based on artificial intelligence combines the artificial intelligence with a spider mobile robot, and adopts a bionic design hand picking method to pick tender tea shoots in a breaking mode, so that famous tea can be picked quickly and efficiently. Famous tea picking harvester based on artificial intelligence has the advantages that: 1. the picking is selective, tender tea shoots are picked in a breaking mode, the old and tender tea shoots are strictly distinguished, the mistaken picking rate, the missing picking rate and the breakage rate are reduced, and various index requirements of famous tea are met. 2. During the working process of the famous tea picking harvester, tea trees and tea tender shoots are not damaged. 3. The famous tea picking harvester can work continuously for 24 hours in all weather without being influenced by factors such as time, weather and the like. 4. The famous tea picking harvester can simultaneously carry out multi-region cooperative work, has high working efficiency and greatly relieves the problem of insufficient labor. 5. The famous tea picking harvester has an autonomous operation function, and can automatically stop operation when reaching the tail end of a tea ridge and send a corresponding signal to a control system. 6. The tea leaf recognition and positioning based on the deep learning algorithm greatly improves the picking precision of the picking system. 7. Based on the tea tender shoot and young sprout cloud decision-making system for 5G transmission, the picking efficiency is greatly improved.

The technical features mentioned above are combined with each other to form various embodiments which are not listed above, and all of them are regarded as the scope of the present invention described in the specification; also, modifications and variations may be suggested to those skilled in the art in light of the above teachings, and it is intended to cover all such modifications and variations as fall within the true spirit and scope of the invention as defined by the appended claims.

Claims (8)

1. A famous tea picking machine based on artificial intelligence recognition is characterized by comprising a power system, a control system, a walking system, an illumination system, a vision system, a picking system, a transmission system, an infrared detection system and a collection box; the power system drives the traveling system to travel forwards under the action of the control system, four two-eye cameras are used as a visual system of the famous tea picking harvester, after image acquisition is completed, the images are uploaded to a control system server through a local or 5G cloud, then the tea leaves are identified and positioned by using an image identification technology and an artificial intelligence method, the picking system is moved and rotated to a proper position through a spider hand robot in the picking system, a tail end picker of the picking system is used for breaking and clamping the fresh tips of the tea leaves, then the tea leaves are conveyed to the conveying system and are conveyed to the collecting box through the conveying system, and the collecting box is stored in a layered mode; the picking machine adopts an aluminum profile to build a frame, the top and the periphery of the frame are sealed by a plate, the side plate is sealed and mainly used for isolating an external light source, and the illumination system is positioned in the frame; four binocular cameras adopted by the vision system are positioned in the frame; the infrared detection system is used for detecting whether the famous tea picking harvester exceeds the working range or not, sending out corresponding signals in time and uploading the signals to the control system, and the control system immediately sends out signals for continuing operation or stopping; the picking system comprises a spider mobile phone robot and a tail end picker; the spider hand robot comprises a static platform, a dynamic platform, a driving motor, a driving arm and a driven arm; the static platform is arranged on a top plate of the frame, the tail end of the static platform is respectively connected with the driving arm and the driven arm through hinges to form a telescopic and deformable triangle, and the tail end picker is arranged at the lower end of the movable platform; the picking device comprises a first picking arm, a second picking arm and a clamping arm, wherein the first picking arm and the second picking arm are arranged oppositely and have a certain height difference, the clamping arm is arranged on the second picking arm, the first picking arm and the second picking arm move in a staggered mode during working, the bud shape integrity of tea tender shoots is guaranteed through breaking operation of specific picking points of the tea tender shoots, and the tea tender shoots are clamped through the clamping arm.

2. The famous tea picking machine according to claim 1, wherein the control system is respectively connected with the walking system, the illumination system, the picking system, the infrared detection system and the transmission system; the control system is connected with the walking system and used for reasonably planning the motion trail of the picking harvester; the control system is connected with the illumination system and is used for controlling the light of the illumination system to be switched on and off; the control system is connected with the picking system and is used for carrying out real-time control management on the spider hand robot and the tail end picker, and regulating and controlling the position and speed parameters of the moving part to enable the mechanical device to operate according to an expected track or specified actions; the control system is connected with the infrared detection system and used for receiving signals sent by the infrared detection system and making corresponding control; the control system is connected with the transmission system and controls whether the transmission system operates or not.

3. The famous tea picking machine as claimed in claim 2, wherein the infrared detection system is positioned at the upper part in front of the machine and is used for detecting whether the famous tea picking harvester exceeds the working range, sending out a corresponding signal in time and uploading the signal to the control system; the collecting box is positioned at the rear part of the famous high-quality tea picking harvester and is used for collecting and storing tea tender shoots transmitted from the transmission system.

4. The famous tea picking machine as claimed in claim 3, wherein the vision system identifies the tea shoots by image recognition technology and artificial intelligence, comprising the following steps:

step 1: shooting tea images of a tea garden, preprocessing the tea images, firstly, selecting clear and definite images of tea, secondly, dividing the selected training set images into a training set and a testing set according to the proportion of 7:3, marking the training set images by using Labelimg software, and storing the training set images in an XML file format;

step 2: after image preprocessing, inputting the marked original images of the tea leaves and the corresponding XML files into a built Faster R-CNN model one by one for training, adjusting various parameters to enable the recognition effect of the model to be optimal, storing the trained model and uploading the model to the cloud for use when a machine picks the model;

and step 3: acquiring target identification, class score, coordinates and time parameters through a trained Faster R-CNN model, and storing the data;

and 4, step 4: after the stored data are preprocessed, the identification and the positioning of the tea leaves in the tea garden are realized.

5. The famous tea picking machine as claimed in claim 4, wherein in the step 2, the established fast R-CNN model specifically comprises the following steps:

step (1): the VGG16 network convolution is used to extract the image features of tea: the input is an image with any size, and the output is a feature map containing extracted features; the VGG16 network convolution consists of 13 convolution layers, 13 ReLU function layers and 4 pooling layers; the specific situation is as follows: the convolutional layer: the method is used for realizing parameter sharing; the size of the image is not changed at the convolutional layer, i.e. the size of the input image O is equal to the size of the output image; the output of the convolutional layer is formula 1:

equation 1:

wherein O is an output image, I is an input image, K is the size of a convolution kernel, P is the size of an amplification matrix, and S is a step length;

the ReLU function layer: ReLU is an activation function that does not change the size of the image; the introduced nonlinearity of the ReLU function is used, so that the output of part of neurons is zero, the interdependence among parameters is reduced, and the occurrence of an overfitting problem is relieved; the pooling layer: for reducing the spatial size of the image, reducing the size of the output image to half the size of the input image;

step (2): after the tea image is subjected to network convolution by VGG16, an image different from an original image, namely a characteristic map, is obtained, and the characteristic map distinguishes the color of a target part from the background so as to highlight the difference of the target;

and (3): the region suggestion network RPN is used for generating region suggestions and transmitting the obtained suggestions to the region of interest for pooling; the specific steps for realizing the RPN of the regional proposal network are as follows:

step A: convolving the feature map output by the VGG16 network by using a 3 × 3 sliding window, and reducing the dimension by using two 1 × 1 sliding windows to reduce the calculation parameters and accelerate the operation speed; wherein, a sliding window of 1 × 1 will give suggestions directly, and another sliding window of 1 × 1 will proceed with the following steps to give suggestions;

and B: after convolution through a 1 × 1 sliding window, parameter adjustment is performed, and the offset between an anchor point frame and a true value is evaluated to make a more accurate suggestion, where the offset is calculated as formula 2:

equation 2:

wherein x and y represent the coordinates of the center point of the true value; w and h represent the width and height, respectively, of the true value; x is the number of_aAnd y_aRepresenting the coordinates of the center point of the anchor point frame; w is a_aAnd h_aRespectively representing the width and height of the anchor frame; delta x and delta y are real values and offset of coordinates of the center point of the anchor point frame; Δ w and Δ h represent the width and height offsets between the true value and the anchor box, respectively;

and C: the softmax classifier screens the offset made in the previous step, and reasonably keeps a plurality of numerical values according to the set threshold parameter, so that a region suggestion of the target frame is given;

step D: drawing a plurality of bounding boxes of the target corresponding to a plurality of numerical values given by the softmax classifier, simultaneously reserving the bounding boxes of the target, and transmitting the bounding boxes of the target to the next step;

step E: according to a non-maximum inhibition method, selecting a plurality of target frames generated in the previous step, and reserving one which can cover the target and has a higher classification score as a suggestion;

step F: integrating the suggestions respectively given by the two sliding windows of 1 multiplied by 1 to obtain a total suggestion, and transmitting the total suggestion to the region of interest for pooling;

step (4), pooling of the region of interest: and meanwhile, a final target position frame is obtained through the prediction of a boundary frame, and a final target prediction score is obtained after class scores are screened through a softmax classifier, so that an image with a target mark frame and a prediction score is obtained.

6. An identification method for famous tea pickers based on artificial intelligence identification as claimed in claim 1, comprising the following steps:

step 1: shooting tea images of a tea garden, preprocessing the tea images, firstly, selecting clear and definite images of tea, secondly, dividing the selected training set images into a training set and a testing set according to the proportion of 7:3, marking the training set images by using Labelimg software, and storing the training set images in an XML file format;

step 2: after image preprocessing, inputting the marked original images of the tea leaves and the corresponding XML files into a built Faster R-CNN model one by one for training, adjusting various parameters to enable the recognition effect of the model to be optimal, storing the trained model and uploading the model to the cloud for use when a machine picks the model;

and step 3: acquiring target identification, class score, coordinates and time parameters through a trained Faster R-CNN model, and storing the data;

and 4, step 4: after the stored data are preprocessed, the identification and the positioning of the tea leaves in the tea garden are realized.

7. The identification method according to claim 6, wherein in the step 2, the established Faster R-CNN model specifically comprises the following steps:

step (1): the VGG16 network convolution is used to extract the image features of tea: the input is an image with any size, and the output is a feature map containing extracted features; the VGG16 network convolution consists of 13 convolution layers, 13 ReLU function layers and 4 pooling layers; the specific situation is as follows:

the convolutional layer: the method is used for realizing parameter sharing; the size of the image is not changed at the convolutional layer, i.e. the size of the input image O is equal to the size of the output image; the output of the convolutional layer is formula 1:

equation 1:

wherein O is an output image, I is an input image, K is the size of a convolution kernel, P is the size of an amplification matrix, and S is a step length;

the ReLU function layer: ReLU is an activation function that does not change the size of the image; the introduced nonlinearity of the ReLU function is used, so that the output of part of neurons is zero, the interdependence among parameters is reduced, and the occurrence of an overfitting problem is relieved; the pooling layer: for reducing the spatial size of the image, reducing the size of the output image to half the size of the input image;

step (2): after the tea image is subjected to network convolution by VGG16, an image different from an original image, namely a characteristic map, is obtained, and the characteristic map distinguishes the color of a target part from the background so as to highlight the difference of the target;

and (3): the region suggestion network RPN is used for generating region suggestions and transmitting the obtained suggestions to the region of interest for pooling;

step (4), pooling of the region of interest: and meanwhile, a final target position frame is obtained through the prediction of a boundary frame, and a final target prediction score is obtained after class scores are screened through a softmax classifier, so that an image with a target mark frame and a prediction score is obtained.

8. The identification method according to claim 7, wherein in the step (3), the specific steps for implementing the regional proposal network RPN are as follows:

step A: convolving the feature map output by the VGG16 network by using a 3 × 3 sliding window, and reducing the dimension by using two 1 × 1 sliding windows to reduce the calculation parameters and accelerate the operation speed; wherein, a sliding window of 1 × 1 will give suggestions directly, and another sliding window of 1 × 1 will proceed with the following steps to give suggestions;

and B: after convolution through a 1 × 1 sliding window, parameter adjustment is performed, and the offset between an anchor point frame and a true value is evaluated to make a more accurate suggestion, where the offset is calculated as formula 2:

equation 2:

wherein x and y represent the coordinates of the center point of the true value; w and h represent the width and height, respectively, of the true value; x is the number of_aAnd y_aRepresenting the coordinates of the center point of the anchor point frame; w is a_aAnd h_aRespectively representing the width and of the anchor boxHigh; delta x and delta y are real values and offset of coordinates of the center point of the anchor point frame; Δ w and Δ h represent the width and height offsets between the true value and the anchor box, respectively;

and C: the softmax classifier screens the offset made in the previous step, and reasonably keeps a plurality of numerical values according to the set threshold parameter, so that a region suggestion of the target frame is given;

step D: drawing a plurality of bounding boxes of the target corresponding to a plurality of numerical values given by the softmax classifier, simultaneously reserving the bounding boxes of the target, and transmitting the bounding boxes of the target to the next step;

step E: according to a non-maximum inhibition method, selecting a plurality of target frames generated in the previous step, and reserving one which can cover the target and has a higher classification score as a suggestion;

step F: and integrating the suggestions respectively given by the two 1 × 1 sliding windows to obtain a total suggestion, and transmitting the total suggestion to the region of interest for pooling.

Images