CN111240195A - Automatic control model training and target object recycling method and device based on machine vision - Google Patents
Automatic control model training and target object recycling method and device based on machine vision Download PDFInfo
- Publication number
- CN111240195A CN111240195A CN202010017699.8A CN202010017699A CN111240195A CN 111240195 A CN111240195 A CN 111240195A CN 202010017699 A CN202010017699 A CN 202010017699A CN 111240195 A CN111240195 A CN 111240195A
- Authority
- CN
- China
- Prior art keywords
- automatic control
- control model
- mechanical arm
- target object
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000012549 training Methods 0.000 title claims abstract description 64
- 238000004064 recycling Methods 0.000 title claims description 27
- 238000011084 recovery Methods 0.000 claims abstract description 33
- 230000006870 function Effects 0.000 claims description 30
- 238000012545 processing Methods 0.000 claims description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 15
- 238000003860 storage Methods 0.000 claims description 15
- 239000013598 vector Substances 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 230000004807 localization Effects 0.000 claims 1
- 210000000078 claw Anatomy 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 23
- 238000004364 calculation method Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 101100045541 Homo sapiens TBCD gene Proteins 0.000 description 3
- 101150093640 SSD1 gene Proteins 0.000 description 3
- 101100111629 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR2 gene Proteins 0.000 description 3
- 102100030290 Tubulin-specific chaperone D Human genes 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000005299 abrasion Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 235000003166 Opuntia robusta Nutrition 0.000 description 1
- 244000218514 Opuntia robusta Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000002537 cosmetic Substances 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 235000012054 meals Nutrition 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 235000021485 packed food Nutrition 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/042—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
Abstract
The invention provides a method and a device for automatically controlling model training and target object recovery based on machine vision, which relate to the technical field of machine vision and comprise the following steps: acquiring target point positioning information and mechanical arm motion parameters, wherein the mechanical arm motion parameters comprise included angles of the mechanical arm along a plurality of axial directions; determining the value of a loss function of the automatic control model according to the target point positioning information and the mechanical arm motion parameters; and adjusting the parameters of the automatic control model according to the value of the loss function until the parameters meet the convergence condition, and finishing the training of the automatic control model. According to the invention, the model is trained by acquiring the positioning information of the object grabbing target point and the motion parameters of the mechanical arm as the data set, so that the trained automatic control model has higher positioning resolving precision. And the trained model does not depend on the specific size of the claw, the accuracy influence caused by the claw with fixed specification can be avoided, and the method has higher flexibility.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to a method and a device for automatic control model training and target object recovery based on machine vision.
Background
With the continuous development of science and technology and the continuous improvement of the living standard of people, various automatic devices emerge in large numbers, and provide various conveniences for the production and the living of people by the characteristics of high efficiency and rapidness. For example, more and more large public restaurants are equipped with a tableware recycling device for automatically recycling tableware, so that the workload of workers for recycling the tableware is reduced.
The existing tableware recovery device is often equipped with a mechanical arm for grabbing tableware to be recovered to a designated place for subsequent treatment. However, the coordinate calculation precision of the traditional low-price mechanical arm product is reduced quickly due to irresistible factors such as abrasion of all parts and integral displacement. And the product must be distributed by high-grade logistics service to ensure the safety of the transportation process, and must be accurately corrected by professionals of developers during installation and debugging. After about 500 grabbing operations are completed, the problems of inaccurate grabbing and reduced working quality of mechanical parts are inevitable, and the application and popularization of the tableware recycling device are seriously influenced.
Disclosure of Invention
To achieve at least some of the above objectives, the present invention provides a method for training an automatic control model based on machine vision, which includes:
acquiring target point positioning information and mechanical arm motion parameters, wherein the mechanical arm motion parameters comprise included angles of the mechanical arm along a plurality of axial directions;
determining the value of a loss function of the automatic control model according to the target point positioning information and the mechanical arm motion parameters;
and adjusting the parameters of the automatic control model according to the value of the loss function until the parameters meet the convergence condition, and finishing the training of the automatic control model.
Further, the target point positioning information is acquired by a camera.
Further, the target point positioning information represents position information of the target point in the camera.
Further, the target point positioning information is represented by a two-dimensional code with a unique identity.
Further, the target point positioning information includes depth information.
Further, the determining the value of the loss function of the automatic control model according to the target point positioning information and the robot arm motion parameter includes:
inputting the target point positioning information and the mechanical arm motion parameters into the automatic control model to obtain a plurality of predicted included angles;
and determining the value of the loss function according to the plurality of predicted included angles and the mechanical arm motion parameters.
Further, the loss function is an L2 loss function.
To achieve the above object, an embodiment of the second aspect of the present invention further provides an automatic control model training device based on machine vision, which includes:
the acquisition module is used for acquiring target point positioning information and mechanical arm motion parameters, wherein the mechanical arm motion parameters comprise included angles of the mechanical arm along a plurality of axial directions;
the processing module is used for determining the value of a loss function of the automatic control model according to the target point positioning information and the mechanical arm motion parameters;
and the training module is used for adjusting the parameters of the automatic control model according to the value of the loss function until the convergence condition is met, and completing the training of the automatic control model.
By using the automatic control model training method or device based on machine vision, the model is trained by acquiring the positioning information of the object grabbing target point and the motion parameters of the mechanical arm as a data set, so that the trained automatic control model has higher positioning resolving precision. And the trained model does not depend on the specific size of the claw, the accuracy influence caused by the claw with fixed specification can be avoided, and the method has higher flexibility.
To achieve the above object, an embodiment of a third aspect of the present invention provides a target object recycling method based on machine vision, including:
acquiring positioning information and target classification information of a target to be recovered;
inputting the positioning information into an automatic control model of a mechanical arm, and determining the rotation angle of the mechanical arm, wherein the mechanical arm is used for recovering the target object to be recovered, and the automatic control model of the mechanical arm is obtained by training by adopting the automatic control model training method based on machine vision;
and controlling the mechanical arm to recover the target object to be recovered according to the target object classification information and the rotation angle.
Further, the positioning information includes position information determined according to a pre-trained target detection model and depth information determined according to a depth camera.
Further, the acquiring the classification information of the target object to be recovered includes:
judging whether the target object to be recovered belongs to a sample with a label;
if the sample belongs to the sample with the label, determining the target object classification information of the target object to be recovered as the label;
and if the target object does not belong to the labeled sample, determining the classification information of the target object according to a weak supervised learning algorithm.
Further, the determining the target object classification information according to a weak supervised learning algorithm comprises:
extracting feature vectors of a plurality of labeled samples into a plurality of corresponding centers;
determining the distances from the feature vectors of the target object to be recovered to a plurality of centers respectively;
and selecting the label corresponding to the center closest to the target object to be recovered as the target object classification information of the target object to be recovered.
Further, after the controlling the mechanical arm to recover the target object to be recovered according to the target object classification information and the rotation angle, the method further includes:
judging whether the mechanical arm is successfully controlled to recover the target object to be recovered and recording the recovery failure times;
and when the recovery failure times exceed a preset threshold value, automatically calibrating the automatic control model of the mechanical arm.
To achieve the above object, an embodiment of a fourth aspect of the present invention provides a target recycling apparatus based on machine vision, including:
the acquisition module is used for acquiring positioning information and target classification information of a target to be recovered;
the processing module is used for inputting the positioning information into an automatic control model of a mechanical arm and determining the rotation angle of the mechanical arm, the mechanical arm is used for recovering the target object to be recovered, and the automatic control model of the mechanical arm is obtained by training by adopting the automatic control model training method based on machine vision;
and the recovery module is used for controlling the mechanical arm to recover the target object to be recovered according to the target object classification information and the rotation angle.
By using the target object recovery method or device based on machine vision, the angle of the mechanical arm needing to rotate is determined according to the positioning information of the target object to be recovered through the trained automatic control model, so that independent data acquisition of equipment can be realized, the structural form is flexibly defined, the control of the mechanical arm is independent of the size of a specific claw, and the method or device has higher flexibility. The invention can more accurately complete the grabbing action according to the classification information of the target object, and improve the efficiency of recovering the target object. The invention does not need to be frequently debugged by professional personnel, and really realizes full-automatic operation.
To achieve the above object, an embodiment of a fifth aspect of the present invention provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a machine vision-based automatic control model training method according to the first aspect of the present invention or implements a machine vision-based object recovery method according to the third aspect of the present invention.
To achieve the above object, an embodiment of a sixth aspect of the present invention provides a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for training a machine vision-based automatic control model according to the first aspect of the present invention or implements the method for recovering a machine vision-based object according to the third aspect of the present invention when executing the program.
The non-transitory computer-readable storage medium and the computing device according to the present invention have similar beneficial effects to the machine vision-based automatic control model training method according to the first aspect of the present invention or the machine vision-based object recovery method according to the third aspect of the present invention, and will not be described in detail herein.
Drawings
FIG. 1 is a schematic diagram of a method for training a machine vision-based automatic control model according to an embodiment of the invention;
FIG. 2 is a schematic diagram of the principle of determining the value of a loss function according to an embodiment of the invention;
FIG. 3 is a schematic flow chart of the optimization step according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an embodiment of an apparatus for training an automatic control model based on machine vision;
FIG. 5 is a schematic diagram of a method for object recovery based on machine vision according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a structure of a target detection model according to an embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating a principle of obtaining classification information of an object according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a weakly supervised learning algorithm according to an embodiment of the present invention;
FIG. 9 is a schematic flow chart of a target recycling method based on machine vision according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of an error accumulation evaluation algorithm according to an embodiment of the present invention;
11a and 11b are schematic diagrams of practical application of the target object recycling method based on machine vision according to the embodiment of the invention;
FIG. 12 is a schematic diagram of a machine vision based object recovery apparatus according to an embodiment of the present invention;
FIG. 13 is a schematic diagram of a computing device according to an embodiment of the invention.
Detailed Description
Embodiments in accordance with the present invention will now be described in detail with reference to the drawings, wherein like reference numerals refer to the same or similar elements throughout the different views unless otherwise specified. It is to be noted that the embodiments described in the following exemplary embodiments do not represent all embodiments of the present invention. They are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the claims, and the scope of the present disclosure is not limited in these respects. Features of the various embodiments of the invention may be combined with each other without departing from the scope of the invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
With the continuous development of science and technology, more and more industrial or commercial scenes need to be equipped with target object recovery units, realize the automatic recovery of target object, improve holistic work efficiency, but at present mainly rely on the manual work to carry out the recovery of target object, inefficiency just causes certain recovery classification error easily because of people's negligence. For example, taking the tableware recycling as an example, the automatic tableware recycling faces several problems:
1. the peak flow is large: often, at the end of a meal, a large number of dishes may be poured into the collection area of the dishes, and if the dishes are not disposed of in time, the dishes may be piled on the table top and cause crowding. This requires the tableware collecting apparatus to have high operation efficiency.
2. Irregular placement: because the placing mode is not specially specified, the user can place the tableware at will, and the tableware is inclined at various angles, stacked and shielded. Therefore, the robot arm for collecting tableware has to have a high degree of freedom and a space for covering various positions. The requirement for identification needs to meet the three-dimensional rotation invariance and also needs to infer and complement the grabbing circle. And determining a reasonable grabbing path to avoid collapse.
3. The items are many in subclasses: tableware is generally classified into five major categories (plate, bowl, chopsticks, fork and spoon), but for example, bowls have many similar sub-categories in appearance, and the improvement of classification fine granularity is required. The inability to put different categories together can add to the burden of returning the cutlery to each window at a later time.
In one embodiment, the position of the robotic arm may be determined based on a coordinate calculation algorithm, which may then be subsequently controlled to grasp the object. However, the coordinate calculation algorithm needs to determine coordinates through mathematical calculation according to accurate values of the arm length, the angle and the like of the mechanical arm. However, the coordinate calculation is easily affected by factors such as component abrasion and displacement, so that the precision based on the coordinate calculation algorithm cannot be guaranteed.
The invention provides a method and a device for automatically controlling model training and target object recovery based on machine vision and machine learning, which can perform self-perception learning according to the positioning information of a target grabbed object and the motion parameters of a mechanical arm, determine the rotation angle of the mechanical arm for different grabbed object target points, do not depend on the specific size of a claw, and have higher flexibility and accuracy. The method has self-correction capability and has the excellent characteristics of high identification precision and strong robustness.
Fig. 1 is a schematic diagram illustrating a method for training an automatic control model based on machine vision according to an embodiment of the present invention, including steps S11-S13.
In step S11, target point positioning information and robot arm motion parameters are acquired, where the robot arm motion parameters include included angles of the robot arm along multiple axial directions. In the embodiment of the invention, the automatic control model is used for controlling the claws at the top end of the mechanical arm to grab the object to be grabbed at the target point by adjusting the mechanical arm to rotate along different axes based on machine vision, so that the automatic control of the mechanical arm is realized, and the object at the target point is grabbed.
In the embodiment of the invention, the target point positioning information is acquired by a camera, and the target point positioning information is represented by a two-dimensional code with a unique identity. For example, the method is imported into an Opencv-consistency-acquuo database, determines the unique identity, sets token (identity authentication temporary token) and graph size, and can generate the autonomous controllable two-dimensional code through a series of image processing algorithms. In this embodiment of the present invention, the target point positioning information represents position information of the target point in the camera. The two-dimensional code is fixed at the position where the grabber is placed, so that the positioning information of the target point can be acquired in the camera. In other embodiments of the present invention, a separate foldable support may be provided to facilitate reading and replacing the two-dimensional code. It can be understood that, if the two-dimensional code is not set at the position of the grabbed point, the corresponding target point positioning information is increased by a corresponding offset for correction adjustment.
In the embodiment of the invention, the target point positioning information comprises depth information and is used for forming three-dimensional information by combining the positioning information of the two-dimensional code, so that the subsequent processing such as calculation control and the like by combining with a mechanical arm coordinate system is facilitated.
The method comprises the steps of enabling a mechanical arm to rotate, paying attention to recording motion control parameters, and calculating a coordinate system of the mechanical arm according to analytic geometrical knowledge and a computer graphics algorithm, so that the original point of the mechanical arm and position information of a claw can be obtained.
In the embodiment of the present invention, a control model of the robot arm is established by polynomial regression according to the positioning information of the target point and the robot arm motion parameter, and a specific example is given below:
wherein x, y and z represent mesh, respectivelyLocation information of punctuation, wx1～wz3、w′x1～w′z3、w″x1～w″z3A parameter representing the automatic control model to be trained, bx、by、bzRespectively, the offset amount. It is understood that the truth value can be more approximated by powers, and the specific form is not limited by the above formula.
In step S12, a value of a loss function of the automatic control model is determined according to the target point positioning information and the robot arm movement parameter. In the embodiment of the invention, the error is assumed to be gaussian noise, and based on the chebyshev approximation theory, the L2 loss function (namely, mean square error) is adopted for optimization. During model training, the parameters can be converged to a suboptimal solution according to probability, wherein f (x) is obtained as the mean value of the sample y, and the expectation of Gaussian noise is u, so that the solution with the maximum probability is obtained. Because the kinematics formula order of the mechanical arm is low, the polynomial decomposition can reach enough precision, and the training effect of the model is ensured.
FIG. 2 is a schematic diagram illustrating a principle of determining a value of a loss function according to an embodiment of the present invention, including steps S21-S22.
In step S21, the target point positioning information and the robot arm motion parameters are input into the automatic control model to obtain a plurality of predicted included angles. In the embodiment of the present invention, the constructed loss function is:
wherein N represents the number of collected target points, edit represents the positioning information of the target points, angle represents the included angle of the mechanical arm along a plurality of axial directions,
representing the estimated value, argminW() The parameter W at which this formula is minimized is shown.
In step S22, a value of the loss function is determined according to the plurality of predicted included angles and the mechanical arm motion parameters. In the embodiment of the invention, according to the obtained predicted included angle and the included angle of the corresponding mechanical arm along the axial direction, the mean square error (namely L2 Loss) of the predicted included angle can be determined, and then the value of the Loss function is determined.
In step S13, the parameters of the automatic control model are adjusted according to the value of the loss function until a convergence condition is satisfied, and the training of the automatic control model is completed. In the embodiment of the invention, the parameter w in the control model is iteratively optimizedx1～wz3、w′x1～w′z3、w″x1～w″z3And when the value of the loss function is minimized, namely the convergence condition is met, completing the training of the automatic control model.
It can be understood that, in the embodiment of the present invention, an optimization step may be further included, and fig. 3 is a schematic flowchart illustrating the optimization step according to the embodiment of the present invention, wherein after the above-mentioned optimization updating of the model parameters is completed, a trained automatic control model is tested by using a part of data reserved in advance as a test set, and an accuracy of the model is verified. If the accuracy rate exceeds a preset threshold value, the model training is considered to be successful, otherwise, the model training can be carried out again to ensure the accuracy of the model training.
By adopting the automatic control model training method based on machine vision, the model is trained by acquiring the positioning information of the object grabbing point and the motion parameters of the mechanical arm as the data set, so that the trained automatic control model has higher positioning resolving precision. And the trained model does not depend on the specific size of the claw, the accuracy influence caused by the claw with fixed specification can be avoided, and the method has higher flexibility.
The embodiment of the second aspect of the invention also provides an automatic control model training device based on machine vision. Fig. 4 is a schematic structural diagram of an automatic control model training apparatus 400 based on machine vision according to an embodiment of the present invention, which includes an obtaining module 401, a processing module 402, and a training module 403.
The obtaining module 401 is configured to obtain target point positioning information and mechanical arm motion parameters, where the mechanical arm motion parameters include included angles of the mechanical arm along multiple axial directions. In this embodiment of the present invention, the target point positioning information is obtained by a camera, and the target point positioning information represents position information of the target point in the camera. The target point positioning information is represented by a two-dimensional code with a unique identity, and the target point positioning information comprises depth information.
The processing module 402 is configured to determine a value of a loss function of the automatic control model according to the target point positioning information and the robot arm motion parameter. The processing module 402 includes a prediction module 4021 and a calculation module 4022 (both not shown in fig. 4), where the prediction module 4021 is configured to input the target point positioning information and the mechanical arm motion parameters into the automatic control model to obtain a plurality of predicted included angles, and the calculation module 4022 is configured to determine a value of the loss function according to the plurality of predicted included angles and the mechanical arm motion parameters.
The training module 403 is configured to adjust parameters of the automatic control model according to the value of the loss function until a convergence condition is met, and complete training of the automatic control model.
The more detailed implementation of each module of the automatic control model training device 400 based on machine vision can be referred to the description of the automatic control model training method based on machine vision of the present invention, and has similar beneficial effects, and will not be described herein again.
The embodiment of the third aspect of the invention provides a target object recycling method based on machine vision. FIG. 5 is a schematic diagram illustrating a method for object recovery based on machine vision according to an embodiment of the present invention, including steps S51-S53.
In step S51, the positioning information and the object classification information of the object to be recovered are acquired. In an embodiment, the object to be recovered includes tableware (e.g., bowls, chopsticks, dinner plates, forks, dinner knives, etc.), object packaging boxes (e.g., beverage bottles, cosmetic packaging boxes, etc.), packaged food, etc. which can be grasped by the robot arm, and the embodiment of the present invention is not limited thereto. In the embodiment of the present invention, the positioning information includes position information and depth information, the position information is determined according to a pre-trained target detection model, and the depth information is determined according to a depth camera.
In the embodiment of the present invention, the pre-trained target Detection model may adopt, for example, a Single Shot Detection (SSD) model using MobileNet as a backbone network. The MobileNet is a lightweight deep neural network capable of operating in embedded computing equipment, and is high in speed and accuracy. Because the network is a lightweight network, the actual requirements of the application scene of target object recovery can be met by using the MobileNet as a feature extraction layer. Fig. 6 is a schematic structural diagram of a target detection model according to an embodiment of the present invention, in which a basic network VGG16 in an SSD target detection framework is replaced by a MobileNet network, 5 convolutional layers (SSD1 to SSD5) are added behind the last convolutional layer, and the 5 layers are extracted for detection, as shown in fig. 6, the 5 layers are extracted in the network, and are SSD1 to SSD 5. SSD1 features have dimensions 19 × 19, SSD2 features have dimensions 10 × 10, SSD3 features have dimensions 5 × 5, SSD4 features have dimensions 3 × 3, and SSD5 features have dimensions 1 × 1. In the embodiment of the invention, the convolution kernels adopting the target detection model have different scaling ratios, and the characteristic diagram is output in the middle, so that the insensitivity of the scale is ensured.
In the embodiment of the invention, the pre-trained target detection model is used for detecting and acquiring the two-dimensional position information of the target object to be recovered, the point cloud picture of the target object to be recovered is acquired through the depth camera, and the coordinate information acquired in the two-dimensional space is aligned to the point cloud picture, so that the three-dimensional space coordinate conversion is realized. The point cloud chart is used for realizing the visualization of coordinate data, getting rid of the dependence on the measurement of accurate values of angles and lengths and improving the positioning accuracy of the target object to be recovered.
FIG. 7 is a schematic diagram illustrating a principle of obtaining classification information of an object according to an embodiment of the present invention, including steps S71-S73.
In step S71, it is determined whether or not the target object to be collected belongs to a labeled specimen. In the embodiment of the invention, the target detection model can be pre-trained by using the training set with the labeling information, or the characteristic extraction is performed on part of samples which may appear in the current use scene, so as to increase the number of classification data sets. It will be appreciated that in practice, some of the objects to be recovered belong to the labelled sample, but there may still be unlabelled samples due to the different shapes of the objects.
In step S72, if the sample belongs to the labeled sample, it is determined that the target classification information of the target to be recovered is the label. In the embodiment of the invention, the feature extraction is carried out on the target object to be recovered according to the target detection model, and if the target object is classified into the existing labeled sample, the label is directly used as the classification of the sample.
In step S73, if the sample does not belong to the labeled sample, the target object classification information is determined according to a weak supervised learning algorithm. FIG. 8 is a schematic diagram illustrating a weak supervised learning algorithm according to an embodiment of the present invention, including steps S81-S83.
In step S81, the feature vectors of the labeled samples are extracted as corresponding centers. In the embodiment of the present invention, the feature vectors of the plurality of samples are extracted as a plurality of centers, respectively, and it is understood that a plurality of feature vectors may also be extracted as a center cluster.
In step S82, distances from the feature vectors of the target object to be recovered to the plurality of centers are determined, respectively. In the embodiment of the present invention, the euclidean distances between the feature vector of the target object to be recovered and the plurality of centers may be used for the determination. It can be understood that the average distance from the feature vector of the target object to be recovered to each central cluster can also be calculated for judgment.
In step S83, the label corresponding to the closest center is selected as the object classification information of the object to be collected. In the embodiment of the invention, according to the idea of cluster analysis, the classification of any target object of a specific type can be realized without any assistance of a user, a large number of samples are not required to be learned in advance, and the system resource and the cost are saved.
Fig. 9 is a schematic flow chart of a target object recycling method based on machine vision according to an embodiment of the present invention, and the embodiment of the present invention is better explained with reference to fig. 5.
In step S52, the positioning information is input into an automatic control model of a robot arm, the rotation angle of the robot arm is determined, the robot arm is used for recovering the target object to be recovered, and the automatic control model of the robot arm is obtained by training with the automatic control model training method based on machine vision. In the embodiment of the invention, the positioning information of the target to be recovered, which is acquired by the target detection model, is input into the trained automatic control model, so that each angle of the mechanical arm needing to be rotated, which is predicted by the model, can be output, and further the subsequent grabbing action is carried out.
In step S53, the robot arm is controlled to recover the object to be recovered according to the object classification information and the rotation angle. In the embodiment of the invention, the mechanical arm can be controlled to rotate correspondingly according to the classification information and the rotation angle of the target object, a reasonable grabbing path is determined according to the grabbing and placing execution business logic, and a grabbing part (such as a claw) is controlled to reclaim the target object to be reclaimed.
As shown in fig. 9, in the embodiment of the present invention, a self-correction step may be further included, in which after the mechanical arm is controlled to recover the target object to be recovered according to the target object classification information and the rotation angle, an error accumulation evaluation algorithm is used for evaluation. Fig. 10 is a schematic diagram illustrating the principle of an error accumulation evaluation algorithm according to an embodiment of the present invention, including steps S101 to S102.
In step S101, it is determined whether the robot arm is successfully controlled to recover the target object to be recovered and the recovery failure times are recorded. In the embodiment of the present invention, after the capture homing service is executed once, whether the task is successfully captured or not is recorded, and it can be understood that the feedback can be derived from two aspects: firstly, the camera recalls the position again and judges whether the target grabbing object (namely the target object to be recovered) moves or not; secondly, the mechanical arm is provided with or without a target. And if the target object does not move or the mechanical arm does not grab the target object to be recovered, the grabbing and homing service is considered to fail.
In step S102, when the recovery failure number exceeds a preset threshold, the robot arm automatic control model is automatically calibrated. In an embodiment of the present invention, the automatic calibration may include re-executing the two-dimensional code positioning training process, or other feedback adjustment procedures, such as a calibration patch.
In the embodiment of the invention, the VPU image processing unit can be used, so that the target object recovery method based on machine vision can be operated without a cloud end, and has better real-time performance.
According to the target object recovery method based on machine vision, the angle of the mechanical arm needing to rotate is determined according to the positioning information of the target object to be recovered through the trained control model, independent data acquisition of equipment can be achieved, the structural form is flexibly defined, the control of the mechanical arm is independent of the size of a specific claw, and the method has higher flexibility. The invention can more accurately complete the grabbing action according to the classification information of the target object, and improve the efficiency of recovering the target object. The invention does not need to be frequently debugged by professional personnel, and really realizes full-automatic operation. The method has the advantages of automatic calibration capability, high identification precision and strong robustness. Fig. 11a and 11b are schematic diagrams illustrating an actual application of the target object recycling method based on machine vision according to the embodiment of the invention. As shown in fig. 11a, the object recycling method based on machine vision according to the embodiment of the present invention can perform the weak supervised learning and classification on objects with various sizes and shapes, and the grabbing accuracy reaches a striking 98.74% in the case of 121 samples. Although the published german artificial intelligence research institute thesis calls that the accuracy rate reaches 99%, the speed of collecting data is faster in the embodiment of the invention, as shown in fig. 11b, accurate grabbing of targets with different shapes and sizes can be achieved only by operating the mechanical arm for 3.6min, manual intervention is not needed, automatic operation is achieved completely, and the method has strong practicability.
The embodiment of the fourth aspect of the invention provides a target object recycling device based on machine vision. Fig. 12 is a schematic structural diagram of a target object recycling apparatus 1200 based on machine vision according to an embodiment of the present invention, including an obtaining module 1201, a processing module 1202, and a recycling module 1203.
The obtaining module 1201 is configured to obtain positioning information and target classification information of a target to be recovered. The obtaining module 1201 includes a target detecting module, configured to obtain the positioning information of the target object to be recovered. The obtaining module 1201 further includes a target classification module 12011 (not shown in the figure), configured to determine whether the target to be recovered belongs to a sample with a label, determine that the target classification information of the target to be recovered is the label if the target belongs to the sample with the label, and determine the target classification information according to a weak supervised learning algorithm if the target does not belong to the sample with the label. It is understood that the object classification module 12011 includes a weak supervised learning module 120111 (not shown in the figure), configured to extract feature vectors of a plurality of labeled samples as a corresponding plurality of centers, determine distances from the feature vectors of the object to be recovered to the plurality of centers, respectively, and select a label corresponding to a center closest to the distance as the object classification information of the object to be recovered. In the embodiment of the present invention, the positioning information includes position information and depth information, the position information is determined according to a pre-trained target detection model, and the depth information is determined according to a depth camera.
The processing module 1202 is configured to input the positioning information into an automatic control model of a mechanical arm, and determine a rotation angle of the mechanical arm, where the mechanical arm is configured to retrieve the target object to be retrieved, and the automatic control model of the mechanical arm is trained by using the automatic control model training method based on machine vision.
The recycling module 1203 is configured to control the mechanical arm to recycle the target object to be recycled according to the target object classification information and the rotation angle.
For a more detailed implementation of each module of the target object recycling apparatus 1200 based on machine vision, reference may be made to the description of the target object recycling method based on machine vision of the present invention, and similar beneficial effects are obtained, and no further description is given here.
Optionally, the apparatus 1200 shown in fig. 12 may further include a training module 1204 (not shown in the figure) for performing the model training method shown in fig. 1.
Optionally, the apparatus 1200 shown in fig. 12 may further include an automatic calibration module 1205 (not shown in the figure) for determining whether to successfully control the robot to recover the target object to be recovered and recording the recovery failure times, and when the recovery failure times exceed a preset threshold, the robot automatic control model performs automatic calibration.
An embodiment of the fifth aspect of the invention proposes a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the machine vision-based automatic control model training method according to the first aspect of the invention or implements the machine vision-based object recovery method according to the third aspect of the invention.
Generally, computer instructions for carrying out the methods of the present invention may be carried using any combination of one or more computer-readable storage media. Non-transitory computer readable storage media may include any computer readable medium except for the signal itself, which is temporarily propagating.
A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages, and in particular may employ Python languages suitable for neural network computing and TensorFlow, PyTorch-based platform frameworks. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
An embodiment of a sixth aspect of the present invention provides a computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method for machine vision based automatic control model training according to the first aspect of the present invention or implementing the method for machine vision based object recovery according to the third aspect of the present invention when executing the program.
The non-transitory computer-readable storage medium and the computing device according to the fifth and sixth aspects of the present invention may be implemented by referring to the contents specifically described in the embodiments of the first aspect or the third aspect of the present invention, and have similar beneficial effects to the automatic control model training method based on machine vision according to the embodiments of the first aspect of the present invention or the target object recovery method based on machine vision according to the embodiments of the third aspect of the present invention, and are not described herein again.
FIG. 13 illustrates a block diagram of an exemplary computing device suitable for use to implement embodiments of the present disclosure. The computing device 12 shown in FIG. 13 is only one example and should not impose any limitations on the functionality or scope of use of embodiments of the disclosure.
As shown in FIG. 13, computing device 12 may be implemented in the form of a general purpose computing device. Components of computing device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally perform the functions and/or methodologies of the embodiments described in this disclosure.
The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, for example, implementing the machine vision-based object retrieving method and the machine vision-based automatic control model training method mentioned in the foregoing embodiments.
The computing device of the invention can be a server or a terminal device with limited computing power.
Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are illustrative and not to be construed as limiting the present invention, and that changes, modifications, substitutions and alterations can be made in the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (16)
1. An automatic control model training method based on machine vision is characterized by comprising the following steps:
acquiring target point positioning information and mechanical arm motion parameters, wherein the mechanical arm motion parameters comprise included angles of the mechanical arm along a plurality of axial directions;
determining the value of a loss function of the automatic control model according to the target point positioning information and the mechanical arm motion parameters;
and adjusting the parameters of the automatic control model according to the value of the loss function until the parameters meet the convergence condition, and finishing the training of the automatic control model.
2. The machine-vision-based automatic control model training method of claim 1, wherein the target point positioning information is acquired by a camera.
3. The machine-vision-based automatic control model training method of claim 2, wherein the target point positioning information represents position information of the target point in the camera.
4. The machine-vision-based automatic control model training method of claim 1, wherein the target point positioning information is represented by a two-dimensional code with a unique identity.
5. The machine-vision-based automatic control model training method according to any one of claims 1-4, characterized in that the target point localization information comprises depth information.
6. The machine-vision-based automatic control model training method of claim 5, wherein the determining the value of the loss function of the automatic control model according to the target point positioning information and the robot arm motion parameters comprises:
inputting the target point positioning information and the mechanical arm motion parameters into the automatic control model to obtain a plurality of predicted included angles;
and determining the value of the loss function according to the plurality of predicted included angles and the mechanical arm motion parameters.
7. The machine-vision-based automatic control model training method of claim 1, wherein the loss function is an L2 loss function.
8. The utility model provides an automatic control model trainer based on machine vision which characterized in that includes:
the acquisition module is used for acquiring target point positioning information and mechanical arm motion parameters, wherein the mechanical arm motion parameters comprise included angles of the mechanical arm along a plurality of axial directions;
the processing module is used for determining the value of a loss function of the automatic control model according to the target point positioning information and the mechanical arm motion parameters;
and the training module is used for adjusting the parameters of the automatic control model according to the value of the loss function until the convergence condition is met, and completing the training of the automatic control model.
9. A target object recycling method based on machine vision is characterized by comprising the following steps:
acquiring positioning information and target classification information of a target to be recovered;
inputting the positioning information into an automatic control model of a mechanical arm, and determining the rotation angle of the mechanical arm, wherein the mechanical arm is used for recovering the target object to be recovered, and the automatic control model of the mechanical arm is obtained by training by adopting the automatic control model training method based on machine vision according to any one of claims 1-7;
and controlling the mechanical arm to recover the target object to be recovered according to the target object classification information and the rotation angle.
10. The machine-vision-based object recycling method of claim 9, wherein the positioning information comprises position information determined according to a pre-trained object detection model and depth information determined according to a depth camera.
11. The machine-vision-based target object recycling method of claim 9, wherein the obtaining target object classification information of the target object to be recycled comprises:
judging whether the target object to be recovered belongs to a sample with a label;
when the target object to be recovered belongs to the sample with the label, determining the target object classification information of the target object to be recovered as the label;
and when the target object to be recovered does not belong to the labeled sample, determining the classification information of the target object according to a weak supervised learning algorithm.
12. The machine-vision-based object recycling method of claim 11, wherein said determining the object classification information according to a weakly supervised learning algorithm comprises:
extracting feature vectors of a plurality of labeled samples into a plurality of corresponding centers;
determining the distances from the feature vectors of the target object to be recovered to a plurality of centers respectively;
and selecting the label corresponding to the center closest to the target object to be recovered as the target object classification information of the target object to be recovered.
13. The target object recycling method based on machine vision according to claim 9, wherein after controlling the mechanical arm to recycle the target object to be recycled according to the target object classification information and the rotation angle, the method further comprises:
judging whether the mechanical arm is successfully controlled to recover the target object to be recovered or not, and recording the recovery failure times;
and when the recovery failure times exceed a preset threshold value, automatically calibrating the automatic control model of the mechanical arm.
14. A machine vision based object recovery apparatus, comprising:
the acquisition module is used for acquiring positioning information and target classification information of a target to be recovered;
a processing module, configured to input the positioning information into an automatic control model of a robot arm, and determine a rotation angle of the robot arm, where the robot arm is used to recover the target object to be recovered, and the automatic control model of the robot arm is obtained by training according to the automatic control model training method based on machine vision as claimed in any one of claims 1 to 7;
and the recovery module is used for controlling the mechanical arm to recover the target object to be recovered according to the target object classification information and the rotation angle.
15. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements a machine vision based automatic control model training method according to any one of claims 1-7 or implements a machine vision based object recovery method according to any one of claims 9-13.
16. A computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the program, implements a machine vision based automatic control model training method according to any one of claims 1-7 or implements a machine vision based object recovery method according to any one of claims 9-13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010017699.8A CN111240195A (en) | 2020-01-08 | 2020-01-08 | Automatic control model training and target object recycling method and device based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010017699.8A CN111240195A (en) | 2020-01-08 | 2020-01-08 | Automatic control model training and target object recycling method and device based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111240195A true CN111240195A (en) | 2020-06-05 |
Family
ID=70869048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010017699.8A Pending CN111240195A (en) | 2020-01-08 | 2020-01-08 | Automatic control model training and target object recycling method and device based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111240195A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113159410A (en) * | 2021-04-14 | 2021-07-23 | 北京百度网讯科技有限公司 | Training method for automatic control model and fluid supply system control method |
| CN116512254A (en) * | 2023-04-11 | 2023-08-01 | 中国人民解放军军事科学院国防科技创新研究院 | Direction-based intelligent control method and system for mechanical arm, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607635A (en) * | 2016-01-05 | 2016-05-25 | 东莞市松迪智能机器人科技有限公司 | Panoramic optic visual navigation control system of automatic guided vehicle and omnidirectional automatic guided vehicle |
| CN109655069A (en) * | 2019-02-03 | 2019-04-19 | 上海允登信息科技有限公司 | A kind of data center machine room robot navigation positioning system |
| CN109702741A (en) * | 2018-12-26 | 2019-05-03 | 中国科学院电子学研究所 | Mechanical arm visual grasping system and method based on self-supervisory learning neural network |
| CN110171001A (en) * | 2019-06-05 | 2019-08-27 | 闽南师范大学 | A kind of intelligent sorting machinery arm system based on CornerNet and crawl control method |
| CN110293552A (en) * | 2018-03-21 | 2019-10-01 | 北京猎户星空科技有限公司 | Mechanical arm control method, device, control equipment and storage medium |
| CN110466924A (en) * | 2019-08-27 | 2019-11-19 | 湖南海森格诺信息技术有限公司 | Rubbish automatic grasping system and its method |
-
2020
- 2020-01-08 CN CN202010017699.8A patent/CN111240195A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607635A (en) * | 2016-01-05 | 2016-05-25 | 东莞市松迪智能机器人科技有限公司 | Panoramic optic visual navigation control system of automatic guided vehicle and omnidirectional automatic guided vehicle |
| CN110293552A (en) * | 2018-03-21 | 2019-10-01 | 北京猎户星空科技有限公司 | Mechanical arm control method, device, control equipment and storage medium |
| CN109702741A (en) * | 2018-12-26 | 2019-05-03 | 中国科学院电子学研究所 | Mechanical arm visual grasping system and method based on self-supervisory learning neural network |
| CN109655069A (en) * | 2019-02-03 | 2019-04-19 | 上海允登信息科技有限公司 | A kind of data center machine room robot navigation positioning system |
| CN110171001A (en) * | 2019-06-05 | 2019-08-27 | 闽南师范大学 | A kind of intelligent sorting machinery arm system based on CornerNet and crawl control method |
| CN110466924A (en) * | 2019-08-27 | 2019-11-19 | 湖南海森格诺信息技术有限公司 | Rubbish automatic grasping system and its method |
Non-Patent Citations (1)
| Title |
|---|
| 陈新泉: "《数据与数据流的聚类、半监督聚类及加权聚类》", 成都：电子科技大学出版社, pages: 52 - 54 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113159410A (en) * | 2021-04-14 | 2021-07-23 | 北京百度网讯科技有限公司 | Training method for automatic control model and fluid supply system control method |
| CN113159410B (en) * | 2021-04-14 | 2024-02-27 | 北京百度网讯科技有限公司 | Training method of automatic control model and fluid supply system control method |
| CN116512254A (en) * | 2023-04-11 | 2023-08-01 | 中国人民解放军军事科学院国防科技创新研究院 | Direction-based intelligent control method and system for mechanical arm, equipment and storage medium |
| CN116512254B (en) * | 2023-04-11 | 2024-01-23 | 中国人民解放军军事科学院国防科技创新研究院 | Direction-based intelligent control method and system for mechanical arm, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11878433B2 (en) | Method for detecting grasping position of robot in grasping object | |
| CN109360226B (en) | Multi-target tracking method based on time series multi-feature fusion | |
| Sui et al. | Sum: Sequential scene understanding and manipulation | |
| CN112837371A (en) | Object grabbing method and device based on 3D matching and computing equipment | |
| CN113450408A (en) | Irregular object pose estimation method and device based on depth camera | |
| CN111428731A (en) | Multi-class target identification and positioning method, device and equipment based on machine vision | |
| CN113927601B (en) | Method and system for realizing precise picking of mechanical arm based on visual recognition | |
| CN109740613B (en) | Visual servo control method based on Feature-Shift and prediction | |
| US20220203547A1 (en) | System and method for improving automated robotic picking via pick planning and interventional assistance | |
| CN112085789A (en) | Pose estimation method, device, equipment and medium | |
| CN111240195A (en) | Automatic control model training and target object recycling method and device based on machine vision | |
| CN115781673A (en) | Part grabbing method, device, equipment and medium | |
| CA3235569A1 (en) | Automated bin-picking based on deep learning | |
| CN116061187B (en) | Method for identifying, positioning and grabbing goods on goods shelves by composite robot | |
| WO2023124734A1 (en) | Object grabbing point estimation method, apparatus and system, model training method, apparatus and system, and data generation method, apparatus and system | |
| Rogelio et al. | Object detection and segmentation using Deeplabv3 deep neural network for a portable X-ray source model | |
| Wang et al. | GraspFusionNet: a two-stage multi-parameter grasp detection network based on RGB–XYZ fusion in dense clutter | |
| CN116051607A (en) | Personnel track tracking method for tobacco shred overhead warehouse | |
| Shao et al. | Combining rgb and points to predict grasping region for robotic bin-picking | |
| CN115730236A (en) | Drug identification acquisition method, device and storage medium based on man-machine interaction | |
| CN115100416A (en) | Irregular steel plate pose identification method and related equipment | |
| Yang et al. | Integrating Deep Learning Models and Depth Cameras to Achieve Digital Transformation: A Case Study in Shoe Company | |
| CN113408429B (en) | Target detection method and system with rotation adaptability | |
| Bağcı et al. | Building height prediction with instance segmentation | |
| US11491650B2 (en) | Distributed inference multi-models for industrial applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200605 |