CN113681563A - Assembling method and system based on double cameras - Google Patents

Assembling method and system based on double cameras Download PDF

Info

Publication number
CN113681563A
CN113681563A CN202111010634.1A CN202111010634A CN113681563A CN 113681563 A CN113681563 A CN 113681563A CN 202111010634 A CN202111010634 A CN 202111010634A CN 113681563 A CN113681563 A CN 113681563A
Authority
CN
China
Prior art keywords
camera
assembly
coordinate system
mechanical arm
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111010634.1A
Other languages
Chinese (zh)
Other versions
CN113681563B (en
Inventor
莫威
田森
刘颖
李东黎
田洪峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Smartstate Technology Co ltd
Original Assignee
Shanghai Smartstate Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Smartstate Technology Co ltd filed Critical Shanghai Smartstate Technology Co ltd
Priority to CN202111010634.1A priority Critical patent/CN113681563B/en
Publication of CN113681563A publication Critical patent/CN113681563A/en
Application granted granted Critical
Publication of CN113681563B publication Critical patent/CN113681563B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J18/00—Arms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/007—Means or methods for designing or fabricating manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B25J19/023—Optical sensing devices including video camera means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1674—Programme controls characterised by safety, monitoring, diagnostic
    • B25J9/1676—Avoiding collision or forbidden zones
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Manipulator (AREA)

Abstract

The embodiment of the specification provides an assembly method and system based on double cameras, which are applied to the technical field of automation, wherein the assembly scheme based on the double cameras comprises the following steps: the first camera is fixedly positioned above the working area, monitors the assembly working area in real time and is responsible for planning the motion path of the mechanical arm; the second camera is positioned on the mechanical arm, accurate positioning measurement within a small range is carried out in assembly, and the mechanical arm assembles an object to be assembled. Through the cooperation work of first camera, arm and second camera, can carry out automatic precision assembly to the high accuracy instrument, can improve the assembly accuracy nature and the assembly efficiency of high accuracy instrument.

Description

Assembling method and system based on double cameras
Technical Field
The invention relates to the technical field of automatic control, in particular to an assembly method and system based on double cameras.
Background
Along with the development of economy and science and technology in recent years, the requirements on the production and assembly efficiency of high-precision instruments are higher and higher, and the construction of a high-quality, high-reliability, high-flexibility and high-intelligence batch assembly system for the high-precision instruments is a necessary trend.
However, the main assembly mode of the existing high-precision instrument still depends on manual operation, and simple process equipment such as a lifting appliance, a support vehicle and the like is used for positioning and supporting, so that the problems that the position and pose adjustment of parts is not quantifiable, the positioning precision is low, the assembly quality seriously depends on the personal skill and experience of operators and the like exist, the functions of new generation high-precision instruments tend to be integrated, the size and the weight are more and more large, the body structure is diversified, the general assembly working condition is more and more complicated, the assembly of the high-precision instrument consumes long time, and the assembly time of a single-unit product of a spacecraft can reach weeks or even months.
Therefore, it is desirable to introduce new assembly techniques to improve the assembly accuracy and efficiency of high precision instruments.
Disclosure of Invention
In view of this, embodiments of the present disclosure provide an assembling method and system for automatically assembling a high-precision instrument based on a dual camera, which can not only automatically assemble a precision instrument (i.e., a high-precision instrument, such as an aerospace instrument), but also improve the assembling accuracy and efficiency of the high-precision instrument.
The embodiment of the specification provides the following technical scheme:
the embodiment of the specification provides an assembling method based on a double camera, which comprises the following steps:
the method comprises the steps that a first camera monitors an assembly working area in real time, and determines a path plan corresponding to a mechanical arm according to a preset path strategy, wherein the path plan is used for enabling the mechanical arm to move to a preset target position in the assembly working area, and the first camera is fixedly installed in the assembly working area;
the mechanical arm moves to the target position according to the path plan;
after the mechanical arm reaches the target position, a second camera carries out assembling positioning measurement on an object to be assembled in the assembling working area so as to determine a target assembling position, wherein the second camera is installed at a preset position on the mechanical arm;
and the mechanical arm assembles the object to be assembled according to the target assembling position.
The present specification also provides an assembly system based on dual cameras, including:
the mechanical arm is used for assembling an object to be assembled;
the first camera is fixedly installed in an assembly working area and used for monitoring the assembly working area in real time and determining a path plan corresponding to the mechanical arm according to a preset path strategy, wherein the path plan is used for enabling the mechanical arm to move to a preset target position in the assembly working area;
the second camera is arranged at a first preset position on the mechanical arm and used for carrying out assembling positioning measurement on the object to be assembled in the assembling working area after the mechanical arm reaches the target position so as to determine a target assembling position;
and the mechanical arm moves to the target position according to the path plan and assembles the object to be assembled according to the target assembly position.
Compared with the prior art, the beneficial effects that can be achieved by the at least one technical scheme adopted by the embodiment of the specification at least comprise:
the automatic assembly mode enables the assembly of the high-precision instrument to be standardized and has repeatability;
the automatic assembly has high positioning precision and high assembly speed, so that the high-precision instrument is changed from the original manual single-piece assembly into automatic batch assembly, and the production efficiency of the high-precision instrument is greatly improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a dual-camera based assembly system provided in an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of coordinate system calibration and path planning in an assembly system based on a dual camera according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a second camera in a dual-camera based assembling system for performing precise positioning measurement according to an embodiment of the present disclosure;
fig. 4 is a flow chart of a dual camera-based assembly method provided by an embodiment of the present disclosure;
fig. 5 is a flowchart of coordinate system calibration in a dual camera-based assembly method according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of coordinate system calibration in an assembly method based on a dual camera according to an embodiment of the present disclosure;
fig. 7 is a flowchart for performing automatic assembly of an aerospace instrument in a dual-camera-based assembly method according to an embodiment of the present disclosure.
Detailed Description
The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
The following description of the embodiments of the present application is provided by way of specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the present application, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number and aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details. The terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features described as being defined as "first," "second," etc., may explicitly or implicitly include one or more of the features. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
At present, aiming at the assembly of high-precision instruments, such as the assembly of aerospace instruments and equipment, in order to ensure the safety and the assembly precision of the high-precision instruments, the high-precision instruments are still assembled in a manual mode. Due to the rapid development of aerospace instruments and equipment, an automatic assembly scheme capable of meeting the requirements of high-precision instruments is urgently needed.
Based on this, the inventor provides a new automatic assembly scheme aiming at the assembly requirement of high-precision instruments: as shown in fig. 1, in the assembling system for automated assembling of a high-precision instrument based on two cameras provided in the embodiments of the present disclosure, the two cameras are a first camera (e.g., a binocular camera) fixedly disposed at a fixed position above an assembling work area (e.g., an assembling factory building) and a second camera (e.g., a 2D camera) mounted on a robot arm, where the first camera is fixedly disposed above the assembling work area, and can perform real-time monitoring on the entire assembling work area over a long distance, for example, is responsible for path planning of a motion path of the robot arm, for example, for interference check on the motion path of the robot arm during assembling, and the second camera on the robot arm is responsible for precise positioning measurement of assembling conditions within a small range during assembling, for example, when the robot arm moves to an assembling position, after the second camera performs precise positioning measurement on a specific assembling position first, and assembling the mechanical arm.
The assembling system for the automatic assembly of the high-precision instrument based on the double cameras, which is provided by the embodiment of the specification, is convenient for the practical assembly application, can be flexibly deployed according to the practical assembly requirement (such as an assembly factory building), can improve the assembly precision and efficiency through the cooperation of the double cameras and the mechanical arm, and is very suitable for the automatic assembly of the high-precision instrument (such as an aerospace instrument).
The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings.
As shown in fig. 1, 2 and 3, embodiments of the present description provide a dual camera-based assembly system that may include: the camera assembly comprises a first camera, a mechanical arm and a second camera, wherein the first camera is fixedly arranged in an assembly working area, the second camera is arranged at a first preset position on the mechanical arm, for example, the first preset position can be a camera support arranged on a flange plate at the tail end of the mechanical arm, and the second camera can be fixed on the camera support.
In implementation, an object to be assembled (such as a high-precision instrument, equipment, a product and the like to be assembled) can be placed on the installation platform in advance, so that the mechanical arm can assemble the object to be assembled conveniently.
First camera fixed mounting is in the assembly work area, for example on the wall in the assembly factory building, makes things convenient for first camera real-time supervision whole assembly work area, for example whether have personnel's activity in the monitoring assembly, for example the motion track of monitoring arm etc..
The first camera can also plan a motion path of the mechanical arm according to a preset path strategy to determine at least one path specification corresponding to the mechanical arm, wherein the path planning is used for enabling the mechanical arm to move to a preset target position in the assembly work area.
For example, depending on the starting position P of the robot arm before assembly1And a certain target position P to be reached2At least one planned path P is obtained after path planning, the path P moving from the starting position to the target position1→P2。
It should be noted that the path policy for performing the path planning may be preset, adjusted, and the like according to application requirements, for example, a mature path planning algorithm is adopted as the path policy, and this is not limited here.
After the planned path is obtained, the mechanical arm can move to a target position according to the path plan, so that the tail end of the mechanical arm (i.e. the end used for assembling the object to be assembled) reaches the target position, and the second camera can further perform accurate positioning measurement conveniently.
After the mechanical arm reaches the target position, the second camera can carry out assembling positioning measurement on the object to be assembled in the assembling working area so as to determine the target assembling position.
After the accurate target assembly position is obtained through accurate positioning measurement of the second camera, the mechanical arm can assemble the object to be assembled according to the target assembly position, for example, the target assembly position is a screw hole, and at this time, the tail end of the mechanical arm can be assembled by a screw through a screw assembling tool.
Therefore, the assembling system for the automatic assembly of the high-precision instrument based on the dual cameras, which is provided by the embodiment of the specification, is convenient for actual assembly and application, and can be flexibly deployed according to actual assembly requirements (such as an assembly factory building), so that the automatic assembly mode can meet the assembly requirements of the high-precision instrument, the assembly of the high-precision instrument is standardized, and the repeatability is achieved; moreover, the high-precision assembling machine can also obtain extremely high positioning precision during automatic assembling through the cooperative work of the double cameras and the mechanical arm, so that the original manual single-piece assembling of the high-precision instrument is changed into automatic batch assembling, the production efficiency of the high-precision instrument is greatly improved, and the high-precision assembling machine is very suitable for automatic assembling of the high-precision instrument (such as an aerospace instrument).
The assembly system is further exemplified by taking the first camera as a binocular vision camera (such as a binocular camera) and the second camera as a 2D (two-dimensional) vision camera (such as a 2D camera) as an example, but those skilled in the art should understand that the binocular camera, the 2D camera, and the like should not constitute a limitation to the technical solution provided in the present specification.
In some embodiments, when the robot arm is far away from the object (such as a part) to be assembled, the binocular camera located above the working area may shoot the working area, and perform path planning on the motion path of the robot arm to obtain at least one planned path P1→P2。
Wherein, P1Is the current position of the robot arm, P2The attitude of the robot arm before the movement may be as shown in fig. 2 as the target position. Planning P according to path on mechanical arm1→P2Reaches the target position P after moving2Thereafter, the pose of the robot arm can be as shown in fig. 3. At this time, the 2D camera can perform accurate pose deviation measurement on an object to be assembled (such as a part), so that an accurate target assembly position is obtained.
In some embodiments, the automatic assembly of the parts may be performed by adjusting the object to be assembled on the mounting platform or adjusting the posture of the robot arm according to the measurement result of the 2D camera, for example, the position of the object to be assembled on the mounting platform deviates from a theoretical position, for example, the mounting position deviates from a theoretical mounting position.
In some embodiments, the binocular camera can perform interference check on the motion process of the mechanical arm in real-time monitoring, and prevent the mechanical arm from colliding with other objects in an assembly work area in the motion process.
In some embodiments, when there is interference, the first camera may feed back coordinate information of the interference position to the robot arm, and the robot arm may control the motion process according to the coordinate information of the interference position, for example, to decelerate the robot arm until the robot arm stops working before moving to the interference position.
For example, in view of the assembly complexity of high-precision instruments (such as aerospace instruments), personnel often enter the threading operation in the assembly process, and at the moment, the binocular camera can detect the working area in real time by shooting the whole working scene, so that the collision between the assembly personnel and the mechanical arm is prevented, and the safety of the personnel, the mechanical arm and the high-precision instruments is guaranteed.
In some embodiments, the 2D camera may make a photogrammetric determination of the accuracy of the fit.
In implementation, after the mechanical arm completes assembly of the object to be assembled, the 2D camera can photograph the target assembly position and detect whether the assembly precision of the target assembly position meets a preset precision threshold.
In some embodiments, the 2D camera can accurately determine whether the screw assembly condition meets the assembly requirements of a high-precision instrument.
In implementation, when the target device position is the screw hole installation position, the 2D camera can obtain information such as the circle center of the contour sideline of the screw and the circle center of the contour sideline of the screw after shooting, and further determine whether the assembly of the screw meets the preset precision according to the distance between the two circle centers, for example, when the distance between the two circle centers is less than or equal to the preset precision threshold, it can be determined that the assembly of the screw meets the assembly requirement of the high-precision instrument.
In some embodiments, a laser sensor may also be employed to assist the 2D camera (i.e., the second camera) in acquiring three-dimensional coordinate data of the target assembly location.
In an implementation, the dual-camera based assembly system further includes a laser sensor (not shown), wherein the laser sensor is mounted at a second preset position (not shown) of the mechanical arm, and is used in cooperation with the 2D vision camera to acquire three-dimensional coordinate data of the target assembly position by acquiring depth information of the target assembly position.
It should be noted that the second preset position may be determined according to practical application requirements, and is not limited herein.
Through the relative position relation of the 2D camera, the laser sensor and the first camera, the three-dimensional coordinate data of the target assembling position can be obtained based on the acquired depth information, so that the assembling can be carried out according to the three-dimensional coordinate data, and the assembling precision is improved.
The present disclosure also provides a dual camera based assembly method, which can be automatically assembled based on the dual camera based assembly system provided in any one of the foregoing embodiments.
As shown in fig. 4, an embodiment of the present disclosure provides a dual camera-based assembly method, which may include:
step S202, the first camera monitors the assembly work area in real time, and determines a path plan corresponding to the mechanical arm according to a preset path strategy.
In an implementation, the path planning is used for moving the robot arm to a preset target position in the assembly work area, wherein the first camera is fixedly installed in the assembly work area, for example, the first camera is fixedly installed on a wall of an assembly plant, so as to globally monitor an assembly operation area below.
And S204, the mechanical arm moves to the target position according to the path planning.
Step S206, when the mechanical arm reaches the targetAfter the position is reached, the second camera carries out assembly positioning measurement on the object to be assembled in the assembly working area so as to determine a target assembly position (such as a position point P identified in the figure)3) Wherein the second camera is mounted at a predetermined position on the robotic arm.
And S208, assembling the object to be assembled by the mechanical arm according to the target assembling position.
Through steps S202 to S208, based on the cooperative work of the first camera, the mechanical arm, and the second camera, the system components (including the first camera, the second camera, and the mechanical arm) required by the automated assembly of the high-precision instrument are simply and rapidly deployed, so that the high-precision automated assembly requirement of the high-precision instrument can be met, the requirement of the high-precision instrument on the assembly precision can be met, the standardization of the automated assembly operation can be realized, the assembly efficiency can be improved, and the method is very suitable for the automated assembly of the high-precision instrument (such as an aerospace instrument).
In some embodiments, a binocular vision camera (such as a binocular camera) may be used as the first camera, and a 2D (two-dimensional) vision camera (such as a 2D camera) may be used as the second camera, so as to simplify the use of the camera system and its calibration operation in the automated assembly, reduce the cost, and improve the deployment flexibility.
In an implementation, the first camera may include a binocular vision camera (e.g., a binocular camera) and the second camera may include a 2D vision camera (e.g., a 2D camera).
In this case, the dual camera system may be calibrated by the following procedure, that is, the dual camera-based assembling method further includes: calibrating the first coordinate system, the second coordinate system and the third coordinate system, and calibrating a conversion relationship among the three coordinate systems, wherein the first coordinate system is the coordinate system of the first camera, the second coordinate system is the coordinate system of the mechanical arm, and the third coordinate system is the coordinate system of the second camera.
In practice, a reference point on the first camera may be calibrated as an origin of a first coordinate system (e.g., a position point O identified in the figure), and directions corresponding to an x-axis, a y-axis, and a z-axis of the first coordinate system may be calibrated.
Preferably, when directions corresponding to the x-axis, the y-axis and the z-axis of the first coordinate system are calibrated, the directions corresponding to the x-axis, the y-axis and the z-axis of the first coordinate system can be calibrated according to an object to be assembled.
The normal direction of the plane of the mounting platform for placing the object to be assembled is taken as the direction of the z axis; the normal direction of a certain side surface (such as a mounting surface) parallel to the z-axis direction in the object to be assembled is taken as the x-axis direction, and finally the y-axis direction can be determined by utilizing a right-hand rule according to the x-axis direction and the z-axis direction.
In practice, a reference point on the end flange of the robotic arm may be designated as the origin of the second coordinate system (e.g., position point O as identified in the figure)1) And calibrating a first coordinate position of the origin of the second coordinate system in the first coordinate system.
For example, the origin of the coordinate system of the robot arm (i.e., the second coordinate system) may be selected to be any location on the robot arm.
Preferably, the origin can be close to the sixth axis of the tail end of the manipulator during selection, namely close to the position of the mounting tool, so that the intermediate transfer error is reduced, and the recognition and grabbing precision is improved.
Preferably, the midpoint of the flange plate at the tail end of the mechanical arm can be selected to be calibrated as the origin of the coordinate system of the mechanical arm
In practice, a reference point on the second camera may be defined as the origin of the third coordinate system (e.g., the position point O identified in the figure)2) And calibrating a second coordinate position of the origin of the third coordinate system in the second coordinate system according to a preset position of the second camera mounted on the mechanical arm.
For example, a point on a camera support for mounting the second camera may be used as the origin of the third coordinate system, and the relative coordinates of the origin of the coordinate system of the second camera in the coordinate system of the robot arm may be determined from the point on the support and the origin of the coordinate system of the robot arm.
In some embodiments, the coordinate system between the robotic arm and the second camera may be calibrated in advance.
In implementation, as shown in fig. 5, the calibrating the conversion relationship between the second coordinate system and the third coordinate system may include:
and step S402, acquiring position coordinates of the three markers in the third coordinate system.
In practice, three markers may be placed in the field of view of the second camera, such as the wells 1-3 identified in FIG. 3 above, where the three markers are placed in the same plane and not in the same line. At this time, the coordinates of the three markers in the second camera, which can be recorded as coordinates (X0, Y0), (X1, Y1), and (X2, Y2), respectively, can be acquired by the second camera.
S404, constructing a parallelogram according to the three position coordinates, and acquiring coordinate coefficients k1 and k2 corresponding to the target point according to the relative relation between the coordinates (Xi, Yi) of the target point in the third coordinate system and the three position coordinates.
In implementation, as shown in fig. 6, a parallelogram is constructed from three of the position coordinates, and then coordinates (Xi, Yi) of the target point in the coordinate system of the second camera can be represented by vectors of two coordinate axes, and coordinate coefficients k1 and k2 are calculated.
Wherein the coordinates of the target point and the coordinate coefficients k1, k2 have a first relation with the three position coordinates as follows:
(Xi，Yi)＝(X0，Y0)+k1[(X1-X0)，(Y1-Y0)]+k2[(X2-X0)，(Y2-Y0)]
the coordinate coefficients k1 and k2 in the first relational expression are representing coefficients of the target point in a non-rectangular coordinate system formed by three position coordinates.
It should be noted that, since the three markers do not change, the coordinate coefficients do not change in the camera coordinate system and the robot arm coordinate system, and therefore, by using this characteristic, the corresponding coordinate coefficient is obtained for each target point, and then the corresponding coordinate point in the coordinate system is obtained by using this coefficient.
Step S406, touching the three markers by using a probe installed at the tail end of the mechanical arm to obtain coordinates (A0, B0, C0), (A1, B1, C1) and (A2, B2, C2) of the three markers in the second coordinate system, and determining coordinates (Ai, Bi, Ci) of the target point in the second coordinate system according to the coordinates of the three markers in the second coordinate system.
And the position (Ai, Bi, Ci) of the target point in the mechanical arm coordinate system can obtain the following second relation between the coordinates (Ai, Bi, Ci) of the target point in the second coordinate system and the coordinates of the three markers in the second coordinate system according to the similar triangle theorem:
(Ai，Bi，Ci)＝(A0，B0，C0)+k1[(A1-A0)，(B1-B0)，(C1-C0)]+k2[(A2-A0)，(B2-B0)，(C2-C0)]
wherein k1 and k2 are coordinate coefficients in the first relational expression.
Therefore, the position (Ai, Bi, Ci) of the target point (Xi, Yi) in the coordinate system of the robot arm (i.e., the second coordinate system) can be obtained in the coordinate system of the second camera (i.e., the third coordinate system).
In some embodiments, while the first camera monitors the assembly work area in real time, the first camera can also perform real-time interference check on the motion path of the mechanical arm, so as to ensure that the posture of the mechanical arm can be updated in time, and avoid collision between the mechanical arm and other objects in the assembly work area.
In implementation, the first camera may perform real-time interference inspection on the mechanical arm in a motion process planned according to the path according to a preset interference strategy, so as to prevent the mechanical arm from colliding with an object in the assembly work area in the motion process.
For example, in real-time monitoring of the assembly area, it may be checked whether a foreign object enters the working area, and if so, it is detected that the robot arm is on the planned path P1→P2Whether the collision with foreign objects occurs during the movement.
In some embodiments, a plurality of three-dimensional shapes, such as a rectangular parallelepiped, a cylinder, etc., corresponding to the mechanical arm for detection may be constructed, so as to facilitate the first camera to perform the interference check quickly.
For example, detecting that a rectangular parallelepiped representing the robot arm is on planned path P1→P2Whether or not to meet in the process of exerciseThe overlapping with the foreign object occurs, and if the overlapping occurs, the foreign object enters the motion path of the mechanical arm and interferes.
Through the real-time interference check, the assembly complex scene of high accuracy instrument (like the space flight instrument) can be ensured, even if someone gets into the threading operation in the assembly process, the whole working scene is still shot to the first camera of accessible, carries out real-time detection to the work area, has ensured the safety of people and arm, prevents that the collision from producing.
In some embodiments, when the first camera detects that the interference position exists, coordinate information of the interference position may be acquired by using the first camera, and the coordinate information of the interference position may be fed back to the robot arm, so that the robot arm may control a motion process of the robot arm according to the coordinate information of the interference position.
For example, when there is interference, information of the interference position may be fed back to the robot arm, and the robot arm may be decelerated until it stops working before moving to the interference position.
In some embodiments, after the mechanical arm completes assembling the object to be assembled, the second camera may perform high-precision inspection on the assembling condition to determine whether the assembling meets the assembling requirement of a high-precision instrument.
In implementation, the second camera can photograph the target assembly position and detect whether the assembly precision of the target assembly position meets a preset precision threshold.
In some embodiments, when the target assembly position is the screw hole installation position, the second camera may measure whether a circle center distance between a circle center of the contour line of the screw and a circle center of the contour line of the screw hole is smaller than a preset accuracy threshold value, so as to perform high-accuracy inspection on the assembly condition of the screw.
In implementation, after assembly is completed, the mechanical arm moves to a second camera (such as a 2D camera) to align to the center position of the screw, and the positional relationship between the outline edge line of the screw and the outline edge line of the screw is measured to determine whether the assembly precision meets the requirement, and if the distance between the centers of the two outline edge lines is less than a certain value, the assembly precision meets the requirement.
The present specification also provides a dual camera based assembly method that allows for precise automated assembly of high precision instruments such as aerospace instruments.
The following is an exemplary description in conjunction with the various embodiments described above.
As shown in fig. 7, the automatic assembly process for the space instrument may include the following steps according to the assembly requirement:
step S602, calibrating the two cameras, for example, calibrating a coordinate system of a first camera (such as a binocular camera), a coordinate system of a second camera (such as a 2D camera), and a coordinate system of the mechanical arm, for example, calibrating a conversion relationship between the coordinate systems;
step S604, determining a conversion relation between a mechanical arm coordinate system and a camera coordinate system, for example, determining a relation between the mechanical arm and a coordinate system of a first camera, a relation between the mechanical arm and a coordinate system of a second camera, and the like;
step S606, utilizing a binocular camera to identify and determine the mechanical arm coordinates P1 of the starting position and the position P2 of the target point, and planning at least one feasible mechanical arm motion path P1→P2；
Step S608, P in the robot arm1→P2In the movement process, the binocular camera acquires images of a working area in real time, monitors whether a foreign object enters the working area, and if so, further judges whether the movement paths of the foreign object and the mechanical arm are overlapped, namely, carries out interference check;
step S610, when the superposition is generated, the interference position is checked, a signal is fed back to the mechanical arm, the mechanical arm is decelerated until the mechanical arm stops working before moving to the interference position, and the mechanical arm starts to work again after the foreign object leaves a working area; normally working if no overlap is generated;
after the robot arm moves to P2, the position P3 of the fitting target point is precisely determined using the 2D camera (i.e., the second camera), and after the position P3 is precisely determined, the robot arm moves to P3 to start the fitting;
and step S614, after the assembly is finished, detecting the position relation of the outline side lines by using the 2D camera, and determining whether the assembly precision meets the requirement.
Through the assembling scheme based on the double cameras provided by the embodiment of the specification, the automatic assembling of the space instrument can be realized, the assembling precision is high, the efficiency is high, the artificial assembling error in the manual assembling is avoided, and an intelligent assembling scheme is provided for the assembling process of the space instrument.
It should be noted that, for the above steps S602 to S614, reference may be specifically made to the relevant descriptions related to the foregoing embodiments, and details are not described here again.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the product embodiments described later, since they correspond to the method, the description is simple, and the relevant points can be referred to the partial description of the method embodiments.
In this specification, various embodiments may be entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware implementations.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A dual camera based assembly method, comprising:
the method comprises the steps that a first camera monitors an assembly working area in real time, and determines a path plan corresponding to a mechanical arm according to a preset path strategy, wherein the path plan is used for enabling the mechanical arm to move to a preset target position in the assembly working area, and the first camera is fixedly installed in the assembly working area;
the mechanical arm moves to the target position according to the path plan;
after the mechanical arm reaches the target position, a second camera carries out assembling positioning measurement on an object to be assembled in the assembling working area so as to determine a target assembling position, wherein the second camera is installed at a preset position on the mechanical arm;
and the mechanical arm assembles the object to be assembled according to the target assembling position.
2. The dual camera-based assembly method of claim 1, wherein the first camera comprises a binocular vision camera and the second camera comprises a 2D vision camera;
the dual camera based assembly method further comprises:
calibrating the first coordinate system, the second coordinate system and the third coordinate system, and calibrating a conversion relation among the three coordinate systems, wherein the first coordinate system is a coordinate system of the first camera, the second coordinate system is a coordinate system of the mechanical arm, and the third coordinate system is a coordinate system of the second camera;
the method comprises the following steps that a certain datum point on a first camera is calibrated as an origin of a first coordinate system, and directions corresponding to an x axis, a y axis and a z axis of the first coordinate system are calibrated;
a certain datum point on a flange plate at the tail end of the mechanical arm is marked as an origin of a second coordinate system, and a first coordinate position of the origin of the second coordinate system in the first coordinate system is marked;
and calibrating a certain reference point on the second camera as an origin of a third coordinate system, and calibrating a second coordinate position of the origin of the third coordinate system in the second coordinate system according to a preset position of the second camera on the mechanical arm.
3. The dual camera-based assembly method as set forth in claim 2, wherein calibrating the transfer relationship between the second coordinate system and the third coordinate system comprises:
acquiring position coordinates of three markers in the third coordinate system, which are respectively marked as coordinates (X0, Y0), (X1, Y1) and (X2, Y2), wherein the three markers are placed in the visual field of the second camera;
constructing a parallelogram according to the three position coordinates, and acquiring coordinate coefficients k1 and k2 corresponding to the target point according to the relative relation between the coordinates (Xi, Yi) of the target point in the third coordinate system and the three position coordinates, wherein the coordinate coefficients k1 and k2, the three position coordinates and the coordinates of the target point have the following first relational expression:
(Xi，Yi)＝(X0，Y0)+k1[(X1-X0)，(Y1-Y0)]+k2[(X2-X0)，(Y2-Y0)]
the coordinate coefficients k1 and k2 in the first relational expression are the representation coefficients of the target point under a non-rectangular coordinate system formed by three position coordinates;
touching three of the markers with a probe mounted at a tip of the robotic arm, obtaining coordinates (a0, B0, C0), (a1, B1, C1) and (a2, B2, C2) of the three markers in the second coordinate system, and determining coordinates (Ai, Bi, Ci) of the target point in the second coordinate system from the coordinates of the three markers in the second coordinate system, wherein the coordinates (Ai, Bi, Ci) of the target point in the second coordinate system and the coordinates (Ai, Bi, Ci) of the three markers in the second coordinate system have a second relationship as follows:
(Ai，Bi，Ci)＝(A0，B0，C0)+k1[(A1-A0)，(B1-B0)，(C1-C0)]+k2[(A2-A0)，(B2-B0)，(C2-C0)]
wherein k1 and k2 are coordinate coefficients in the first relational expression.
4. The dual camera-based assembly method as claimed in claim 1, further comprising:
and the first camera performs real-time interference inspection on the mechanical arm in the motion process planned according to the path according to a preset interference strategy so as to prevent the mechanical arm from colliding with an object in the assembly working area in the motion process.
5. The dual camera-based assembly method of claim 4, wherein when the first camera detects the presence of an interference location, the dual camera-based assembly method further comprises:
the first camera acquires coordinate information of an interference position, and feeds the coordinate information of the interference position back to the mechanical arm;
and the mechanical arm controls the motion process according to the coordinate information of the interference position.
6. The dual camera-based assembly method of claim 4, further comprising:
creating a plurality of target three-dimensional shapes, wherein the target three-dimensional shapes are used for representing the appearance of a mechanical arm containing the second camera;
the first camera performs real-time interference inspection on the mechanical arm in the motion process planned according to the path, and the method comprises the following steps:
and the first camera detects that the target cuboid performs real-time interference inspection according to the motion process of the path planning.
7. The dual camera-based assembly method as claimed in any one of claims 1 to 6, wherein after the arm completes the assembly of the object to be assembled, the dual camera-based assembly method further comprises:
and the second camera shoots the target assembly position and detects whether the assembly precision of the target assembly position meets a preset precision threshold value.
8. The dual-camera based assembly method of claim 7, wherein when the target assembly position is a screw hole installation position, the detecting whether the assembly accuracy of the target assembly position satisfies a preset accuracy threshold includes: and measuring whether the circle center distance between the circle center of the outline sideline of the screw and the circle center of the outline sideline of the screw hole is smaller than a preset precision threshold value.
9. A dual camera based assembly system, comprising:
the mechanical arm is used for assembling an object to be assembled;
the first camera is fixedly installed in an assembly working area and used for monitoring the assembly working area in real time and determining a path plan corresponding to the mechanical arm according to a preset path strategy, wherein the path plan is used for enabling the mechanical arm to move to a preset target position in the assembly working area;
the second camera is arranged at a first preset position on the mechanical arm and used for carrying out assembling positioning measurement on the object to be assembled in the assembling working area after the mechanical arm reaches the target position so as to determine a target assembling position;
and the mechanical arm moves to the target position according to the path plan and assembles the object to be assembled according to the target assembly position.
10. The dual camera-based assembly system of claim 9, wherein the first camera comprises a binocular vision camera and the second camera comprises a 2D vision camera;
the assembling system based on the double cameras further comprises a laser sensor, wherein the laser sensor is installed at a second preset position of the mechanical arm and used for being matched with the 2D visual camera to obtain three-dimensional coordinate data of the target assembling position by obtaining depth information of the target assembling position.
CN202111010634.1A 2021-08-31 2021-08-31 Assembling method and system based on double cameras Active CN113681563B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111010634.1A CN113681563B (en) 2021-08-31 2021-08-31 Assembling method and system based on double cameras

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111010634.1A CN113681563B (en) 2021-08-31 2021-08-31 Assembling method and system based on double cameras

Publications (2)

Publication Number Publication Date
CN113681563A true CN113681563A (en) 2021-11-23
CN113681563B CN113681563B (en) 2022-10-18

Family

ID=78584372

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111010634.1A Active CN113681563B (en) 2021-08-31 2021-08-31 Assembling method and system based on double cameras

Country Status (1)

Country Link
CN (1) CN113681563B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114770505A (en) * 2022-04-29 2022-07-22 清华大学 Gripping device, assembly equipment and control method thereof
CN114905511A (en) * 2022-05-12 2022-08-16 南京航空航天大学 Industrial robot assembly error detection and precision compensation system calibration method
CN115648224A (en) * 2022-12-22 2023-01-31 北京钢铁侠科技有限公司 Mechanical arm grabbing method based on double-depth camera recognition and positioning
CN117140540A (en) * 2023-08-04 2023-12-01 上海智元新创技术有限公司 Fastening system and pick-up fastening method of fastening system

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6481083B1 (en) * 1999-06-25 2002-11-19 Burke E. Porter Machinery Company Robotic apparatus and method for mounting a vale stem on a wheel rim
JP2010240757A (en) * 2009-04-01 2010-10-28 Honda Motor Co Ltd Workpiece aligning system and workpiece moving method
CN102729253A (en) * 2011-04-15 2012-10-17 株式会社安川电机 Robot system and processed object manufacturing method
CN202752803U (en) * 2012-08-08 2013-02-27 苏州逸美德自动化科技有限公司 Assembling machine based on double vision positioning
CN210731548U (en) * 2019-08-30 2020-06-12 珠海格力智能装备有限公司 Assembling equipment
CN112405490A (en) * 2020-11-25 2021-02-26 集美大学 Flexible assembly robot with autonomous navigation and positioning functions
CN113146172A (en) * 2021-03-08 2021-07-23 中国科学院宁波材料技术与工程研究所 Multi-vision-based detection and assembly system and method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6481083B1 (en) * 1999-06-25 2002-11-19 Burke E. Porter Machinery Company Robotic apparatus and method for mounting a vale stem on a wheel rim
JP2010240757A (en) * 2009-04-01 2010-10-28 Honda Motor Co Ltd Workpiece aligning system and workpiece moving method
CN102729253A (en) * 2011-04-15 2012-10-17 株式会社安川电机 Robot system and processed object manufacturing method
CN202752803U (en) * 2012-08-08 2013-02-27 苏州逸美德自动化科技有限公司 Assembling machine based on double vision positioning
CN210731548U (en) * 2019-08-30 2020-06-12 珠海格力智能装备有限公司 Assembling equipment
CN112405490A (en) * 2020-11-25 2021-02-26 集美大学 Flexible assembly robot with autonomous navigation and positioning functions
CN113146172A (en) * 2021-03-08 2021-07-23 中国科学院宁波材料技术与工程研究所 Multi-vision-based detection and assembly system and method

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114770505A (en) * 2022-04-29 2022-07-22 清华大学 Gripping device, assembly equipment and control method thereof
CN114905511A (en) * 2022-05-12 2022-08-16 南京航空航天大学 Industrial robot assembly error detection and precision compensation system calibration method
CN114905511B (en) * 2022-05-12 2023-08-11 南京航空航天大学 Industrial robot assembly error detection and precision compensation system calibration method
CN115648224A (en) * 2022-12-22 2023-01-31 北京钢铁侠科技有限公司 Mechanical arm grabbing method based on double-depth camera recognition and positioning
CN117140540A (en) * 2023-08-04 2023-12-01 上海智元新创技术有限公司 Fastening system and pick-up fastening method of fastening system

Also Published As

Publication number Publication date
CN113681563B (en) 2022-10-18

Similar Documents

Publication Publication Date Title
CN113681563B (en) Assembling method and system based on double cameras
US10357879B2 (en) Robot zero-point calibration device and method
KR102280663B1 (en) Calibration method for robot using vision technology
JP6280525B2 (en) System and method for runtime determination of camera miscalibration
JP4021413B2 (en) Measuring device
EP1288754B1 (en) System and method for producing an assembly by directly implementing three-dimensional computer-aided design component definitions
US20190242971A1 (en) Relative Object Localization Process for Local Positioning System
CN110370286A (en) Dead axle motion rigid body spatial position recognition methods based on industrial robot and monocular camera
CN108827155B (en) Robot vision measurement system and method
Jiang et al. A measurement method for robot peg-in-hole prealignment based on combined two-level visual sensors
CN107883929A (en) Monocular vision positioner and method based on multi-joint mechanical arm
CN101261118A (en) Rapid automatized three-dimensional appearance on-line measurement method and system based on robot
Morozov et al. Assessing the accuracy of industrial robots through metrology for the enhancement of automated non-destructive testing
CN110202575A (en) A kind of robot target path accuracy compensation method for commercial measurement
CN110355788A (en) Moving operation machine National People's Congress scale space high-precision on-line proving system
JP2007160486A (en) Off-line programming device
CN109579766A (en) A kind of product shape automatic testing method and system
CN109341532A (en) A kind of part coordinate scaling method based on structure feature towards automatic assembling
CN103759635A (en) Scanning measurement robot detection method allowing precision to be irrelevant to robot
CN113146613B (en) Three-dimensional self-calibration device and method for D-H parameters of industrial robot
Zhang et al. Online adaptive measurement and adjustment for flexible part during high precision drilling process
CN114459345B (en) Aircraft fuselage position and posture detection system and method based on visual space positioning
CN109773589B (en) Method, device and equipment for online measurement and machining guidance of workpiece surface
CN111376272B (en) Robot measurement path planning method for three-dimensional scanning process of shell structure
Jian et al. On-line precision calibration of mobile manipulators based on the multi-level measurement strategy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant