CN113920322A - Modular robot kinematic chain configuration identification method and system - Google Patents

Modular robot kinematic chain configuration identification method and system Download PDF

Info

Publication number
CN113920322A
CN113920322A CN202111228925.8A CN202111228925A CN113920322A CN 113920322 A CN113920322 A CN 113920322A CN 202111228925 A CN202111228925 A CN 202111228925A CN 113920322 A CN113920322 A CN 113920322A
Authority
CN
China
Prior art keywords
module
center point
kinematic chain
input image
bounding box
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111228925.8A
Other languages
Chinese (zh)
Other versions
CN113920322B (en
Inventor
李伟昌
黄尚樱
梁梓熙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202111228925.8A priority Critical patent/CN113920322B/en
Publication of CN113920322A publication Critical patent/CN113920322A/en
Application granted granted Critical
Publication of CN113920322B publication Critical patent/CN113920322B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a modular robot kinematic chain configuration identification method and a system, wherein the method comprises the steps of obtaining a modular robot image to obtain an input image; inputting an input image into a preset target detection model for multi-module identification to obtain a module central point position, a module type and a module bounding box size, wherein the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit; and processing the position of the center point of the module, the type of the module and the size of a module enclosure frame based on a nearest point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain. The system comprises: the device comprises an input module, a detection module and a configuration identification module. By using the method and the device, the kinematic chain configuration identification of the robot can be realized under the condition that no additional detection component is required to be installed on the robot, and the robustness to a noise environment is higher. The invention can be widely applied to the field of visual identification.

Description

Modular robot kinematic chain configuration identification method and system
Technical Field
The invention relates to the field of visual identification, in particular to a modular robot kinematic chain configuration identification method and system.
Background
The robot is widely regarded by modern large-scale industrial production due to high efficiency, accuracy and low cost, and the modular robot system has high expansibility and reconfigurability and can be applied to different fields in various configurations. However, configuration recognition of a modular robotic system is a time-consuming and less accurate task, especially for modular robotic systems lacking sensing devices.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a method and a system for identifying a kinematic chain configuration of a modular robot, where the algorithm is a semi-supervised learning method, the kinematic chain configuration of the robot can be identified without installing an additional detection component to the robot, and the method and the system have high robustness to a noise environment.
The first technical scheme adopted by the invention is as follows: a modular robot kinematic chain configuration identification method comprises the following steps:
acquiring a modular robot image to obtain an input image;
inputting an input image into a preset target detection model for multi-module identification to obtain a module central point position, a module type and a module bounding box size;
the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit;
and processing the position of the center point of the module, the type of the module and the size of a module enclosure frame based on a nearest point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain.
Further, the output unit includes:
a central point predicting section for predicting a position of a central point of a module in an input image and a type of the module;
an offset predicting section for predicting an offset of a center point position of a block in an input image;
and a bounding box prediction section for predicting a size of a bounding box of the block in the input image.
Further, the step of inputting the input image into a preset target detection model for multi-module recognition to obtain a module central point position, a module type and a module enclosure frame specifically includes:
inputting an input image into a preset target detection model;
performing primary feature extraction processing on the input image based on a primary feature extraction unit to obtain an intermediate feature tensor;
performing depth feature fusion on the intermediate feature tensor based on the feature fusion unit to obtain a high-level feature tensor;
sampling the high-level feature tensor and adjusting the size to obtain a feature map;
respectively inputting the characteristic diagram into a central point prediction part, an offset prediction part and a surrounding frame prediction part to obtain module central point output, module central point offset correction quantity output and module surrounding frame output;
obtaining the position of the center point of the module and the type of the module based on the output of the center point of the module and the output of the offset correction of the center point of the module;
and obtaining the size of the module bounding box based on the output of the module bounding box.
Further, the module center point output is represented as follows:
Figure BDA0003315278410000021
in the above formula, R represents the ratio of the feature map to the input image size, C represents the number of module types, w1Is the width of the image, h1Is the height of the image.
Further, the method also comprises a construction step of a central point prediction part, which specifically comprises the following steps:
constructing a confidence label of a module central point;
confidence of center point of c-class module in (x, y) position
Figure BDA0003315278410000022
Acquiring a training image;
decomposing the real center point p (x, y) in the training image into low-resolution center points
Figure BDA0003315278410000023
Offset from center point
Figure BDA0003315278410000024
Based on low resolution center point
Figure BDA0003315278410000025
And confidence level
Figure BDA0003315278410000026
The center point prediction part is trained.
Further, the mean square error is taken as a loss function of the prediction part of the training center point, and the formula is as follows:
Figure BDA0003315278410000027
in the above formula, the ExRepresenting a mathematical expectation operator, said Yh；x,y,cIndicates a confidence label, LhRepresenting the centroid prediction loss function, x, y representing the position of the training real centroid, and c representing the class.
Further, the method also comprises a construction step of an offset prediction part, which specifically comprises the following steps:
offset based on center point
Figure BDA0003315278410000028
And the L1 loss function trains the bounding box prediction part;
the L1 loss function is
Figure BDA0003315278410000029
Said LoRepresents a center point offset operator, said N represents a total number of modules, | represents an L1 distance operator,
Figure BDA00033152784100000210
represents the predicted value of the center point at the (x, y) position.
Further, the method also comprises a construction step of a bounding box prediction part, which specifically comprises the following steps:
to be provided with
Figure BDA00033152784100000211
To indicate the kind as ckThe module detects the coordinates of the upper left corner and the lower right corner of a square frame in an image to obtain the coordinates of the center point of the square frame
Figure BDA0003315278410000031
And the size of the enclosure frame
Figure BDA0003315278410000032
Taking the height h and the width w of the bounding box s of all the modules in all the categories in the training image as the values of two feature maps respectively, and corresponding the low-resolution bounding box sizes w and h of the module with the central point at the given position (x, y) in the feature maps;
the bounding box prediction part is trained based on the L1 loss function.
Further, the method for searching based on the nearest neighbor processes the position of the module center point, the module type and the size of the module enclosure frame, determines the module connection relationship, and completes the step of identifying the kinematic chain configuration, which specifically includes:
constructing a strong connection graph by taking the central points of all modules in all categories in the input image as nodes;
taking a main control module as a starting point and trying to connect all modules in sequence to obtain a linked list with the minimum weight, wherein the weight is the distance between two central points;
determining the connection relation of all modules according to the intersection and parallel ratio between the central point position and the surrounding frames;
and finishing the identification of the kinematic chain type configuration.
The second technical scheme adopted by the invention is as follows: a modular robot kinematic chain configuration identification system comprising:
the input module is used for acquiring the image of the modular robot to obtain an input image;
the system comprises a detection module, a detection module and a processing module, wherein the detection module is used for inputting an input image into a preset target detection model for multi-module identification to obtain a module central point position, a module type and a module bounding box size, and the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit;
and the configuration identification module is used for processing the position of the center point of the module, the type of the module and the size of a module enclosure frame based on a nearest neighbor point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain.
The method and the system have the beneficial effects that: the method adopts the idea of target detection, uses a deep learning method to obtain the robot module information in a given image, and carries out configuration matching based on a nearest neighbor searching mode to find out the current configuration. The robot kinematic chain configuration recognition method based on the multi-dimensional motion detection can realize the kinematic chain configuration recognition of the robot without installing an additional detection component for the robot, and has high robustness for a noise environment.
Drawings
FIG. 1 is a flow chart illustrating the steps of a modular robot kinematic chain configuration identification method of the present invention;
FIG. 2 is a block diagram of a modular robot kinematic chain configuration identification system according to the present invention;
FIG. 3 is a schematic representation of a configuration identified in accordance with an embodiment of the present invention;
FIG. 4 is a schematic illustration of an identification process according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the experimental validation process of the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
The invention provides a chain type modular robot kinematic configuration identification method based on visual identification, which uses a target detection neural network to complete module identification of different module types and surrounding frames thereof in a modular robot in an image, and then an algorithm realizes the kinematic chain identification by searching the intersection and comparison mode of a nearest module central point and the surrounding frames thereof.
Referring to fig. 1 and 4, the present invention provides a modular robot kinematic chain configuration identification method, including the following steps:
s1, acquiring a modular robot image to obtain an input image;
s2, inputting the input image into a preset target detection model for multi-module recognition to obtain a module central point position, a module type and a module bounding box size;
the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit;
specifically, the feature fusion units are connected in series by adopting a tree structure (HDA), feature information between a shallow layer and a deep layer is effectively combined to encourage the model to learn richer combinations, so that more feature hierarchical structures are expanded, residual connection is introduced into each HDA output to prevent the model from generating a deep degradation problem, and the condition that the gradient of the model disappears or the gradient explodes in the training process can be effectively prevented.
And S3, processing the position of the center point of the module, the type of the module and the size of the module enclosure frame based on the nearest neighbor point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain.
Specifically, the RRMS is a modular robot system having a highly integrated and centralized control system, and the modules of the RRMS may be divided into control modules and function modules according to the definition of the control system. For a given RGB photograph
Figure BDA0003315278410000041
Figure BDA0003315278410000042
Wherein w1Is the width of the image, h1The number 3 characterizes the number of eigenchannels of the image tensor, for the height of the image. We extract high-level semantic information from a given image tensor by a feature extractor h (-), the feature extractor h
Figure BDA0003315278410000043
In general, since the feature extractor has a down-sampling function, there is an inequality w1≥w2，h1≥h2. Then, we will use the output module to map the high-level semantic information z to get the output
Figure BDA0003315278410000044
Fig. 4(a) shows a given input image, fig. 4(b) shows that the module center point position, the module type, and the module bounding box size are obtained, fig. 4(c) shows that the module center point position, the module type, and the module bounding box size are processed based on the nearest neighbor search method to determine the module connection relationship, and fig. 4(d) shows the result of the kinematic chain configuration recognition.
Further as a preferred embodiment of the method, the output unit includes:
a central point predicting section for predicting a position of a central point of a module in an input image and a type of the module;
an offset predicting section for predicting an offset of a center point position of a block in an input image;
and a bounding box prediction section for predicting a size of a bounding box of the block in the input image.
Specifically, we name these three outputs as the center point output in turn
Figure BDA0003315278410000051
Bounding box size output
Figure BDA0003315278410000052
And center point offset correction output
Figure BDA0003315278410000053
As a preferred embodiment of the method, the step of inputting the input image into a preset target detection model for multi-module recognition to obtain a module center point position, a module type, and a module enclosure frame specifically includes.
S21, inputting the input image into a preset target detection model;
s22, performing primary feature extraction processing on the input image based on the primary feature extraction unit to obtain an intermediate feature tensor;
s23, performing depth feature fusion on the intermediate feature tensor based on the feature fusion unit to obtain a high-level feature tensor;
s24, sampling the high-level feature tensor and adjusting the size to obtain a feature map;
s25, inputting the feature graph into a central point prediction part, an offset prediction part and a surrounding frame prediction part respectively to obtain module central point output, module central point offset correction quantity output and module surrounding frame output;
specifically, for an image X containing the RRMS module, it is first converted into an intermediate feature tensor H with four times down-sampling by the preliminary feature extraction module1And then, the feature tensor sequentially carries out deep fusion of features through three feature fusion units so as to extract a feature tensor H with high-level semantic information. Then we perform successive 2 times up-sampling process on H to restore the 16 times down-sampled feature map to 4 times down-sampled size, so as to prevent the accuracy from decreasing due to too small size of the output feature map. Finally, the 4 times down-sampled feature maps are respectively used as the input of three output modules, and the output which finally contains the detection result is obtained
Figure BDA0003315278410000054
S26, obtaining the position of the module center point and the type of the module based on the module center point output and the module center point offset correction quantity output;
specifically, since the minimum scale of the prediction result is 4 pixels, an offset prediction part needs to be separately constructed for correction, that is, the position of the module center point is based on the position of the center point after the offset correction.
And S27, obtaining the size of the module bounding box based on the output of the module bounding box.
Further as a preferred embodiment of the method, in order to make the model have the function of predicting the positions of the center points of the plurality of modules and the types of the modules, let the center point output be expressed as follows:
Figure BDA0003315278410000055
in the above formula, R represents the ratio of the feature map to the input image size, C represents the number of module types, i.e. the feature map of each channel output by the center point predicts the position of the center point of the module of one category, respectively, w1Is the width of the image, h1Is the height of the image.
Under ideal conditions, the center point output of the model will be located at the center point of the correct class of small modules
Figure BDA0003315278410000056
And the other positions are
Figure BDA0003315278410000061
Further, as a preferred embodiment of the method, the method further comprises a step of constructing a central point prediction part, which specifically comprises:
based on Gaussian kernel
Figure BDA0003315278410000062
Constructing a confidence label of a module central point, and enabling a standard deviation sigma to be a constant;
c type module center point in (x, y) positionUpper confidence level
Figure BDA0003315278410000063
Acquiring a training image;
decomposing the real center point p (x, y) in the training image into low-resolution center points
Figure BDA0003315278410000064
Offset from center point
Figure BDA0003315278410000065
Based on low resolution center point
Figure BDA0003315278410000066
And confidence level
Figure BDA0003315278410000067
The center point prediction part is trained.
Further, as a preferred embodiment of the method, the loss function of the prediction part of the training center point is taken as the mean square error, and the formula is expressed as follows:
Figure BDA0003315278410000068
in the above formula, the ExRepresenting a mathematical expectation operator, said Yh；x,y,cIndicates a confidence label, LhRepresenting the centroid prediction loss function, x, y representing the position of the training real centroid, and c representing the class.
Further, as a preferred embodiment of the method, the method further comprises a step of constructing an offset prediction part, which specifically comprises:
offset based on center point
Figure BDA0003315278410000069
And the L1 loss function trains the bounding box prediction part;
predicting only the presence of small module locations in a statistical true tagValue, the L1 loss function is
Figure BDA00033152784100000610
Said LoRepresents a center point offset operator, said N represents a total number of modules, | represents an L1 distance operator,
Figure BDA00033152784100000611
represents the predicted value of the center point at the (x, y) position.
Specifically, for the true center point p of the target (x, y), we decompose it into two parts: low resolution center point
Figure BDA00033152784100000612
Offset from center point
Figure BDA00033152784100000613
The two parts respectively correspond to the center point output
Figure BDA00033152784100000614
And center point offset correction output
Figure BDA00033152784100000615
And p ═ p' + δ p. For center point output
Figure BDA00033152784100000616
In other words, we will express the confidence of the center point of the c-class module at the (x, y) location in the form of a heat map
Figure BDA00033152784100000617
And the abscissa x and the ordinate y of the central point offset δ p of all the modules in all the categories are used as central point offset correction quantity to be output
Figure BDA00033152784100000618
And the coordinates (x, y) of the two feature maps correspond to the module center point offsets δ x and δ y of the center point at the position respectively.
Further, as a preferred embodiment of the method, the method further includes a step of constructing a bounding box prediction part, which specifically includes:
to be provided with
Figure BDA0003315278410000071
To indicate the kind as ckThe module detects the coordinates of the upper left corner and the lower right corner of a square frame in an image to obtain the coordinates of the center point of the square frame
Figure BDA0003315278410000072
And the size of the enclosure frame
Figure BDA0003315278410000073
Taking the height h and the width w of the bounding box s of all the modules in all the categories in the training image as the values of two feature maps respectively, and corresponding the low-resolution bounding box sizes w and h of the module with the central point at the given position (x, y) in the feature maps;
training the bounding box prediction part based on the L1 loss function, i.e.
Figure BDA0003315278410000074
In order to unify the relative magnitude of loss values of different links, two hyper-parameters lambda are addedsAnd λoI.e. by
L＝Lp+λo·Lo+λs·Ls
By the module identification method, we obtain the category c, the central point p and the bounding box of all the small modules in the graph
Figure BDA0003315278410000075
Further, as a preferred embodiment of the method, the step of processing the module center point position, the module type and the module enclosure frame size based on the nearest neighbor point search method, determining the module connection relationship, and completing the identification of the kinematic chain configuration specifically includes:
s31, constructing a strong connection graph by taking the central points of all modules in all categories in the input image as nodes;
s32, determining the connection relation of all modules according to the intersection ratio between the central point position and the surrounding frames;
s33, determining a starting point and trying to connect all modules in sequence to obtain a linked list with the minimum weight, wherein the weight is the distance between two central points;
and S34, finishing the identification of the kinematic chain type configuration, and referring to the identification result in FIG. 3.
Specifically, due to the problem of the shooting angle, the distance difference between the modules is small, and at this time, if the search is directly performed according to the distance between the two central points, an unsatisfactory result may be obtained. We heuristically used the magnitude of the intersection ratio IoU between two blocks to make an auxiliary decision, where IoU is the ratio of the intersection of two bounding boxes to the union of two bounding boxes. IoU is close to 1 as the bounding boxes of the two modules overlap more; otherwise, it is close to 0. IoU is taken as the next joint in the robot configuration, so that the accuracy of the prediction result can be ensured to the maximum extent;
in a specific implementation, we use breadth-first search to accomplish this: and (5) taking the control module (F) as a root node, and traversing other modules which are not in the linked list in sequence for screening. The screening criteria were: the module is the shortest distance from the central point of the linked list tail module, and the module with the largest intersection ratio IoU is used as the new linked list tail module. When the distances between the modules are different from the intersection ratio, or when abnormal behaviors such as the distance of the central point is far larger than the size of the bounding box of the linked list tail module (corresponding to the identified RRMS module which is not in the moving chain) are found, the algorithm refuses to identify and tries to prompt the user to perform manual configuration identification.
Experimental verification was carried out with a specific example:
we randomly picked a photograph and visualized the result as shown in fig. 5. In the actual search, we directly construct two adjacency matrixes based on all modules, namely between every two modulesIntersection ratio IoU matrix HIoUAnd center point spacing HdWhere the subscript (i, j) indicates that the result is given by index i and j, so that the two adjacent matrices are symmetric matrices. For the results shown in fig. 5, there are:
Figure BDA0003315278410000081
Figure BDA0003315278410000082
since we artificially select the master control module (F) as the starting point of the linked list, the serial number of the master control module is 0, as can be obtained from fig. 5 (c). And from HdAnd HIoUIt can be seen that the sequence number 0 module is closest to the sequence number 2 module, and both have the largest intersection ratio IoU, so we add the sequence number 2 module to the linked list. Repeating the above process, and finally obtaining a linked list as follows: [0214]. It can be easily found that the serial number of the linked list is consistent with the configuration of the real module even in the case of large background noise (such as reflection).
Experimental results show that the method can realize the identification of the kinematic chain configuration of the robot under the condition that no additional detection component is required to be installed on the robot, and has high robustness for a noise environment.
As shown in fig. 2, a modular robot kinematic chain configuration recognition system includes:
the input module is used for acquiring the image of the modular robot to obtain an input image;
the system comprises a detection module, a detection module and a processing module, wherein the detection module is used for inputting an input image into a preset target detection model for multi-module identification to obtain a module central point position, a module type and a module bounding box size, and the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit;
and the configuration identification module is used for processing the position of the center point of the module, the type of the module and the size of a module enclosure frame based on a nearest neighbor point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain.
The contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1. A modular robot kinematic chain configuration identification method is characterized by comprising the following steps:
acquiring a modular robot image to obtain an input image;
inputting an input image into a preset target detection model for multi-module identification to obtain a module central point position, a module type and a module bounding box size;
the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit;
and processing the position of the center point of the module, the type of the module and the size of a module enclosure frame based on a nearest point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain.
2. The modular robot kinematic chain configuration identification method according to claim 1, characterized in that the output unit comprises:
a central point predicting section for predicting a position of a central point of a module in an input image and a type of the module;
an offset predicting section for predicting an offset of a center point position of a block in an input image;
and a bounding box prediction section for predicting a size of a bounding box of the block in the input image.
3. The modular robot kinematic chain configuration identification method according to claim 2, wherein the step of inputting the input image into a preset target detection model for multi-module identification to obtain a module center point position, a module type and a module enclosure frame specifically comprises:
inputting an input image into a preset target detection model;
performing primary feature extraction processing on the input image based on a primary feature extraction unit to obtain an intermediate feature tensor;
performing depth feature fusion on the intermediate feature tensor based on the feature fusion unit to obtain a high-level feature tensor;
sampling the high-level feature tensor and adjusting the size to obtain a feature map;
respectively inputting the characteristic diagram into a central point prediction part, an offset prediction part and a surrounding frame prediction part to obtain module central point output, module central point offset correction quantity output and module surrounding frame output;
obtaining the position of the center point of the module and the type of the module based on the output of the center point of the module and the output of the offset correction of the center point of the module;
and obtaining the size of the module bounding box based on the output of the module bounding box.
4. The modular robot kinematic chain configuration identification method of claim 3, characterized in that the module center point output is represented as follows:
Figure FDA0003315278400000011
in the above formula, R represents the ratio of the feature map to the input image size, C represents the number of module types, w1Is the width of the image, h1Is the height of the image.
5. The modular robot kinematic chain configuration identification method according to claim 4, further comprising a central point prediction part construction step, which specifically comprises:
constructing a confidence label of a module central point;
confidence of center point of c-class module in (x, y) position
Figure FDA0003315278400000021
Acquiring a training image;
decomposing the real center point p (x, y) in the training image into low-resolution center points
Figure FDA0003315278400000022
Offset from center point
Figure FDA0003315278400000023
Based on low resolution center point
Figure FDA0003315278400000024
And confidence level
Figure FDA0003315278400000025
The center point prediction part is trained.
6. The modular robot kinematic chain configuration identification method according to claim 5, characterized in that the mean square error is used as a loss function of the predicted part of the training center point, and the formula is as follows:
Figure FDA0003315278400000026
in the above formula, the ExRepresenting a mathematical expectation operator, said Yh；x，y，cIndicates a confidence label, LhRepresenting the centroid prediction loss function, x, y representing the position of the training real centroid, and c representing the class.
7. The modular robot kinematic chain configuration identification method according to claim 6, further comprising a construction step of an offset prediction part, which specifically comprises:
offset based on center point
Figure FDA0003315278400000027
And the L1 loss function trains the bounding box prediction part;
the L1 loss function is
Figure FDA0003315278400000028
Said LoRepresents a center point offset operator, said N represents a total number of modules, | represents an L1 distance operator,
Figure FDA0003315278400000029
represents the predicted value of the center point at the (x, y) position.
8. The modular robot kinematic chain configuration identification method according to claim 7, further comprising a building step of a bounding box prediction part, which specifically comprises:
Figure FDA00033152784000000210
to indicate the kind as ckThe module detects the coordinates of the upper left corner and the lower right corner of a square frame in an image to obtain the coordinates of the center point of the square frame
Figure FDA00033152784000000211
And the size of the enclosure frame
Figure FDA00033152784000000212
Figure FDA00033152784000000213
Taking the height h and the width w of the bounding box s of all the modules in all the categories in the training image as the values of two feature maps respectively, and corresponding the low-resolution bounding box sizes w and h of the module with the central point at the given position (x, y) in the feature maps;
the bounding box prediction part is trained based on the L1 loss function.
9. The modular robot kinematic chain configuration identification method according to claim 8, wherein the step of processing the module center point position, the module type and the module bounding box size based on the nearest neighbor point search method, determining the module connection relationship, and completing the kinematic chain configuration identification specifically comprises:
constructing a strong connection graph by taking the central points of all modules in all categories in the input image as nodes;
determining the connection relation of all modules according to the intersection and parallel ratio between the central point position and the surrounding frames;
determining a starting point and trying to connect all modules in sequence to obtain a linked list with the minimum weight, wherein the weight is the distance between two central points;
and finishing the identification of the kinematic chain type configuration.
10. A modular robot kinematic chain configuration identification system, comprising:
the input module is used for acquiring the image of the modular robot to obtain an input image;
the system comprises a detection module, a detection module and a processing module, wherein the detection module is used for inputting an input image into a preset target detection model for multi-module identification to obtain a module central point position, a module type and a module bounding box size, and the preset target detection model comprises a preliminary feature extraction unit, a feature fusion unit and an output unit;
and the configuration identification module is used for processing the position of the center point of the module, the type of the module and the size of a module enclosure frame based on a nearest neighbor point searching method, determining the connection relation of the modules and finishing the identification of the configuration of the kinematic chain.
CN202111228925.8A 2021-10-21 2021-10-21 Modular robot kinematic chain configuration identification method and system Active CN113920322B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111228925.8A CN113920322B (en) 2021-10-21 2021-10-21 Modular robot kinematic chain configuration identification method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111228925.8A CN113920322B (en) 2021-10-21 2021-10-21 Modular robot kinematic chain configuration identification method and system

Publications (2)

Publication Number Publication Date
CN113920322A true CN113920322A (en) 2022-01-11
CN113920322B CN113920322B (en) 2024-06-14

Family

ID=79242239

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111228925.8A Active CN113920322B (en) 2021-10-21 2021-10-21 Modular robot kinematic chain configuration identification method and system

Country Status (1)

Country Link
CN (1) CN113920322B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108115654A (en) * 2017-12-27 2018-06-05 清华大学 Become drive motion chain and the parallel robot containing the kinematic chain containing V belt translation
CN109829893A (en) * 2019-01-03 2019-05-31 武汉精测电子集团股份有限公司 A kind of defect object detection method based on attention mechanism
CN110378325A (en) * 2019-06-20 2019-10-25 西北工业大学 A kind of object pose recognition methods during robot crawl
CN110503112A (en) * 2019-08-27 2019-11-26 电子科技大学 A kind of small target deteection of Enhanced feature study and recognition methods
US20200293050A1 (en) * 2017-09-22 2020-09-17 Lg Electronics Inc. Mobile robot and method of controlling the same
US20210064876A1 (en) * 2019-08-27 2021-03-04 Ricoh Company, Ltd. Output control apparatus, display control system, and output control method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200293050A1 (en) * 2017-09-22 2020-09-17 Lg Electronics Inc. Mobile robot and method of controlling the same
CN108115654A (en) * 2017-12-27 2018-06-05 清华大学 Become drive motion chain and the parallel robot containing the kinematic chain containing V belt translation
CN109829893A (en) * 2019-01-03 2019-05-31 武汉精测电子集团股份有限公司 A kind of defect object detection method based on attention mechanism
CN110378325A (en) * 2019-06-20 2019-10-25 西北工业大学 A kind of object pose recognition methods during robot crawl
CN110503112A (en) * 2019-08-27 2019-11-26 电子科技大学 A kind of small target deteection of Enhanced feature study and recognition methods
US20210064876A1 (en) * 2019-08-27 2021-03-04 Ricoh Company, Ltd. Output control apparatus, display control system, and output control method

Also Published As

Publication number Publication date
CN113920322B (en) 2024-06-14

Similar Documents

Publication Publication Date Title
CN107851174B (en) Image semantic annotation equipment and method, and generation method and system of image semantic annotation model
CN106780631B (en) Robot closed-loop detection method based on deep learning
CN110619059B (en) Building marking method based on transfer learning
CN112132014B (en) Target re-identification method and system based on non-supervised pyramid similarity learning
US11410327B2 (en) Location determination apparatus, location determination method and computer program
CN113743675B (en) Construction method and system of cloud service QoS deep learning prediction model
CN111507222A (en) Three-dimensional object detection framework based on multi-source data knowledge migration
CN107423747A (en) A kind of conspicuousness object detection method based on depth convolutional network
CN110281949B (en) Unified hierarchical decision-making method for automatic driving
Xu et al. Vision-based multi-level synthetical evaluation of seismic damage for RC structural components: a multi-task learning approach
CN113688797A (en) Abnormal behavior identification method and system based on skeleton extraction
CN112800934A (en) Behavior identification method and device for multi-class engineering vehicle
CN113516133A (en) Multi-modal image classification method and system
CN115147418A (en) Compression training method and device for defect detection model
CN112381730B (en) Remote sensing image data amplification method
CN114120045A (en) Target detection method and device based on multi-gate control hybrid expert model
CN114627397A (en) Behavior recognition model construction method and behavior recognition method
CN113920322A (en) Modular robot kinematic chain configuration identification method and system
JP3251840B2 (en) Image recognition device
CN110647917A (en) Model multiplexing method and system
CN112001388B (en) Method for detecting circular target in PCB based on YOLOv3 improved model
JP2018147103A (en) Model learning device, controlled variable calculation device, and program
CN113343918A (en) Power equipment identification method, system, medium and electronic equipment
CN111546327A (en) Method, apparatus and computer program for determining a motion or trajectory of a robot
CN111652102A (en) Power transmission channel target object identification method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant