CN109895099A - A kind of flight mechanical arm visual servo grasping means based on physical feature - Google Patents
A kind of flight mechanical arm visual servo grasping means based on physical feature Download PDFInfo
- Publication number
- CN109895099A CN109895099A CN201910241810.9A CN201910241810A CN109895099A CN 109895099 A CN109895099 A CN 109895099A CN 201910241810 A CN201910241810 A CN 201910241810A CN 109895099 A CN109895099 A CN 109895099A
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- flight
- unmanned plane
- joint
- speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The flight mechanical arm visual servo grasping means based on physical feature that the present invention provides a kind of, extraction and rear end visual servo including front end natural feature points resolve, pass through the matching with desired character point of realtime graphic, the homography matrix transformation relation between image can be obtained, and then the affine transformation for utilizing homography matrix, can find out corner location of the angle point of desired image in realtime graphic；It is poor that the real-time corner location and desired corner location information obtained using front end is made, and obtains the servo velocity of camera, finally carries out speed control to unmanned plane and mechanical arm；Present invention could apply to long distance transportations to carry task, solve the problems, such as the visual servo of prosthetic label, and the crawl of mechanical arm is efficiently, accurately, therefore it has very big application value.
Description
Technical field
The present invention relates to unmanned plane field more particularly to a kind of flight mechanical arm visual servo crawls based on physical feature
Method.
Background technique
With the rapid development of robot technology, obtain widely answering in industrial manufacture, military operation, domestic life
With.No matter Military Application, civilian, multi-rotor unmanned aerial vehicle transport plays the role of critically important, they can take off vertically, land, hang
Stop.Gradually, multi-rotor unmanned aerial vehicle is not only limited to transport aspect, and compound unmanned plane will become aerial operation machine of new generation
People will replace the mankind to complete dangerous, complex environment aerial operation task.
In recent years, scientific research personnel wishes that unmanned function goes the object in crawl environment, therefore birth with arm as people
Flight mechanical arm is given birth to.Flight mechanical arm is a complicated under-actuated systems, while the multi-degree-of-freemechanical mechanical arm carried is being controlled
Industrial machinery arm is different from system, the whole world is all in Primary Study at present.Due to the flight mechanical arm in most of environment
Movement and crawl must be remain stationary in robot body or stable state under just can be carried out, this greatly limits robots
Movement, while in manipulator motion interference can be generated to unmanned plane, and visual servo can be very good to inhibit various interference, it is real
Now accurately grab.The country only has less pertinent literature about flight mechanical arm at present, though there is small part correlation unmanned plane machine
The patent of invention of tool arm, but it is difficult to find that the crawl of flight mechanical arm visual servo is relevant.The target of usual visual servo is people
Work marker, needs preset in advance, can not carry out friendly interaction with environment, not meet real scene.Most of researcher is only
Consideration is grabbed in the stroke range of mechanical arm, and the distance of unmanned plane potential range object is greater than threshold value in practice,
It is grabbed by the movement of mechanical arm merely, is unable to complete task.The present invention is exactly proposed according to this demand based on certainly
The right visual servo of feature and the joint visual servo strategy of multitask.
" grasping means of a kind of unmanned plane and its view-based access control model " (publication number: CN107139178A).The invention provides
A kind of unmanned plane, including unmanned plane body, mechanical arm, steering engine control joint and camera, the mechanical arm are fixed on the nothing
The lower section of man-machine body, steering engine control joint are connected to the upper front part of the unmanned plane body, and the steering engine control is closed
Section is single-degree-of-freedom, and steering engine control joint is connect with the camera, is demarcated to each joint of mechanical arm, the present invention
A kind of grasping means of unmanned plane view-based access control model is provided, the control in steering engine control joint is carried out with PD control device, so that bowing
It faces upward, on yaw direction, in unmanned plane or target movement, video camera can be accurately aimed at object, carry out pitching, yaw chases after
Track.This method camera is mounted in fuselage, and the scope of activities of camera is small, is not that visual servo is carried out to mechanical arm, but sharp
With the position orientation relation of camera and object, the inverse kinematics of mechanical arm is resolved, can not accomplish real-time.The freedom of mechanical arm simultaneously
It spends low, can only move in a plane, free-position can not be reached.
" a kind of Visual servoing control method in UAV Maneuver target locating " (publication number:
CN105353772A).The invention is by establishing earth coordinates, body coordinate system, camera coordinate system, image coordinate system, machine
Body the earth interim coordinate system carries out target according to the imaging sequence of target by the relationship between the coordinate system of above-mentioned foundation
The calculating of the attitude angle given value of the attitude angle given value and course line tracking of positioning and target following, completes Visual servoing control.
The camera of single fixation is only used only, its advantage is that significantly reducing load without tracking holder and Laser Distance Measuring Equipment
Volume and cost, improve the concealment of scouting.But camera can not be move freely.Limit its visual range.The hair
The bright flight control for relating only to unmanned plane, is not directed to the servo crawl of flight mechanical arm.
" a kind of flight mechanical arm system and control method based on device for force feedback and VR perception " (publication number:
CN109164829A).The invention includes unmanned plane part and earth station part, and unmanned plane part includes unmanned plane body, double
Mesh camera, airborne computer, mechanical arm, control system and visual tag.Control system includes flight master control and mechanical arm master control, machine
It carries computer and control system is mounted on unmanned plane body；Earth station part includes VR aobvious, device for force feedback and ground station owner
The flight control instruction received from ground station host and mechanical arm control instruction are sent to control system by machine, airborne computer
System.The present invention is perceived by VR and display technology, observes the stereo scene in front of unmanned plane in real time with the first visual angle, more smart
It observes quasi-ly, position target object, reduce the difficulty of subsequent mechanical arm control.Using force feedback equipment control mechanical arm fortune
It is dynamic, technology is grabbed with flight mechanical arm, in real time, accurately adjusts mechanical arm position.The invention is mentioned by the way that VR equipment is artificial
For feedback, can not carry out grabbing object from primary feedback.The present invention is not directed to flight mechanical arm visual servo crawl problem.
" a kind of rotor flying mechanical arm system and algorithm based on dynamic center of gravity compensation " (publication number:
CN108248845A).The system includes rotor flying platform, imaging sensor, connection frame, mechanical arm system, system controller
With ground station control device；The rotor flying platform includes rotor craft and flight controller；Described image sensor peace
Mounted in the front lower place of the rotor flying platform；The connection frame is one piece of mechanical plate, for being fixedly connected with the rotor flying
Platform, imaging sensor and mechanical arm system；The mechanical arm system is mounted on the underface of rotor flying platform；The system
Controller is mounted on the surface of rotor flying platform；Nothing is used between the ground station control device and the system controller
Line mode is communicated.The invention devises a kind of center of gravity compensation controller, the disturbance primarily directed to mechanical arm to unmanned plane
Problem.The invention be not directed to how to carry out it is efficient, accurately grab problem.
Summary of the invention
To solve the above problems, the invention proposes a kind of flight mechanical arm visual servo crawl side based on physical feature
Method is extracted by front end natural feature points, is not necessarily to any handmarking's object by the object of servo, it is thus only necessary to shoot in advance
Desired image containing object；Rear end visual servo is resolved using based on the visual servo based on image, passes through machine
The servo velocity of camera, can be transmitted to unmanned plane by the velocity transformation relationship that people learns and mechanical arm carries out speed control.In addition,
By the way that camera to be arranged on the robotic arm, and the visual servo task of unmanned plane is done when remote, do in short distance
The visual servo task of mechanical arm realizes the joint Visual servoing control of multitask, is able to carry out efficient, accurate crawl, tool
Body summary of the invention is as follows:
A kind of flight mechanical arm visual servo grasping means based on physical feature, it is characterised in that: including unmanned plane sheet
Body is provided with mechanical arm pedestal in the drone body, is provided with mechanical arm on the mechanical arm pedestal；The machinery
Multiple rotary joints are set in arm, a steering engine is set in each rotary joint；It is provided in the end of the mechanical arm
Clamping jaw and camera；Control system is provided in the drone body, the control system controls the unmanned plane, machinery
The movement of arm and clamping jaw, the camera and the control system are electrically connected；The flight mechanical arm visual servo crawl side
Method includes the following steps:
Step 1: desired image of the shooting containing object, artificial frame take any rectangular area in object, record lower frame
Take region in the location of pixels of desired image angle point, the position for taking region in realtime graphic for finding frame detects in region
Characteristic point be denoted as desired character point；
Step 2: the camera arrests realtime graphic, the realtime graphic that the control system arrests the camera
Carry out characteristic point detection and matching；
Step 3: by the matching of realtime graphic and desired character point, the homography matrix transformation that can be obtained between image is closed
And then system utilizes the affine transformation of homography matrix, can find out corner location of the angle point of desired image in realtime graphic；
Step 4: it is poor to be made using the real-time corner location and desired corner location information that obtain, obtains the camera
Servo velocity；
Step: 5: by the velocity transformation relationship of robotics, the servo velocity of the camera can be transmitted to institute
It states unmanned plane and the mechanical arm carries out speed control.
Preferably, the central axis of the camera is parallel with the central axis of the clamping jaw.So the clamping of the clamping jaw is more
Accurately.
Preferably, the characteristic point uses ORB characteristic point.
Preferably, improved ORB characteristic point algorithm is as follows:
Step 2.1: input picture；
Step 2.2: Gaussian image pyramid；
Step 2.3: quadtree mesh divides；
Step 2.4: band direction FAST angle point grid；
Step 2.5: redundancy angle point is rejected；
Step 2.6: feature point description.
Preferably, the mechanical arm is capable of providing pitch angle and yaw angle.So that object is always in visual field center.
Preferably, the mechanical arm includes the first joint, second joint, third joint, the 4th joint, the 5th joint and the
Six joints；The end of the mechanical arm is the 6th joint, when the distance of unmanned plane distance objective object is greater than threshold value, only controls nothing
The speed in the 5th joint of the linear velocity of man-machine x, y, z axis direction, the angular speed rotated along the z-axis direction and mechanical arm；When
When the distance of unmanned plane distance objective object is less than threshold value, the speed of mechanical arm is only controlled.Multitask servo grabs mechanical arm
Range is more extensive, more accurately.
Preferably, the servo velocity of the camera is transmitted to the unmanned plane and the mechanical arm in the step 5
Calculating process it is as follows:
Inertial coodinate system O that flight mechanical arm defines, body coordinate system b, mechanical arm base coordinate system a, mechanical arm tail end
Actuator coordinate system e, camera photocentre coordinate system c, object centre coordinate system t, expectation crawl position coordinate system d, ob=[x y
z]TAnd RbIndicate position and rotation of the unmanned plane coordinate system relative to inertial coodinate system,In the form of Eulerian angles
Indicate the direction of unmanned plane, q=[q1 q2 q3 q4 q5 q6]TIndicate the joint angle in each joint of mechanical arm.Next to flying machine
The differential kinematics derivation of tool arm entirety is briefly illustrated that the speed of mechanical arm tail end is transported equal to unmanned plane in flight mechanical arm
The sum of the speed that the raw speed of movable property and flight manipulator motion generate:
ve=ve_a+ve_b (1)
Wherein veIndicate mechanical arm tail end final speed, ve_aIndicate the end of arm speed that manipulator motion generates, ve _bIndicate the end of arm speed that unmanned plane movement generates.
It is available according to the differential kinematics of mechanical arm:
Wherein J (q) indicates the geometry Jacobian matrix of mechanical arm,It is neat to indicate that mechanical arm positive kinematics solve
Rotational component in submatrix,Indicate the angular speed in each joint, J2Variable after indicating abbreviation.
Robot arm end effector speed goes under unmanned plane coordinate system and is expressed as:
WhereinIndicate the rotational component of unmanned plane pedestal relative mechanical arm end,Indicate stem-winder end with respect to nothing
The translational component of man-machine pedestal, S (*) indicate that vector turns antisymmetric matrix, vbIt is the movement velocity of unmanned plane, J1After indicating abbreviation
Variable.
Formula (2) and formula (3) are merged:
Multi-rotor unmanned aerial vehicle itself is a underactuated control system, controlled amounts be three degree of freedom linear velocity component and
Along the angular velocity of rotation of aircraft z-axis.Angular velocity of rotation passes through the inertial measuring unit of aircraft itself on remaining two axis
It obtains and is adjusted as the internal feedback ring of stabilized flight, therefore formula (4) can be write as:
ve=Jv=Jcovco+Jucvuc (5)
Wherein JucTo correspond to uncontrollable component in joint Jacobian matrix JColumn vector, JcoFor joint
Remove the Jacobian matrix of uncontrollable part in Jacobian matrix J,For controllable component.
During practical flight, multi-rotor unmanned aerial vehicle remains steady state, i.e. pitch angle and roll angle is close
In zero, therefore uncontrollable part hardly impacts system, formula (5) can approximation write as:
ve=Jv=Jcovco+Jucvuc≈Jcovco (6)
Setting speed allocation matrix is W ∈ R10*n, number of the n depending on depending on needing control amount, vu∈Rn*1For control
The vector of amount, for practical problem vco
Extract corresponding vu, formula (6) can further be write as:
ve=JcoWvu=Juvu (7)
When the distance of unmanned plane distance objective object is greater than threshold value, the Speed allotment matrix in formula (7) can be write as:
When the distance of unmanned plane distance objective object is less than threshold value, the Speed allotment matrix in formula (7) can be write as:
Beneficial effects of the present invention are as follows:
1, the present invention is not available manual features problem for visual servo under common scenarios, proposes to be based on physical feature
Visual servo grasping means, Conventional visual servo needs to attach handmarking's object, and in practical application, subject to conditions, nothing
Method posts handmarking's object in advance, and the method for physical feature need to only shift to an earlier date frame and take target object area, can realize object
Visual servo crawl.
2, the present invention is directed to mechanical arm visual servo overtravel problem, proposes the joint Visual servoing control plan of multitask
Slightly, when the distance of unmanned plane distance objective object is greater than threshold value (threshold value can be formulated according to actual needs), pass through speed
The speed of servo is transmitted to unmanned plane and carries out visual servo by allocation matrix, in order to guarantee that camera can see object always,
Pitch angle and yaw angle servo are carried out using the joint of mechanical arm；When unmanned plane enters the working space of mechanical arm, only to machinery
Arm carries out visual servo operation, and it is more efficient, accurate to make to grab.
Present invention could apply to long distance transportation carry task, it is particularly possible to apply disaster relief scene or intricately
The place that the people such as shape environment are hard to reach solves the problems, such as the visual servo of prosthetic label, therefore it has very big application
Value.
Detailed description of the invention
Fig. 1 is the mechanical arm configuration figure of present invention flight；
Fig. 2 is that the present invention is based on the algorithm block diagrams that the flight mechanical arm of the visual servo algorithm of physical feature grabs；
Fig. 3 is that the present invention improves ORB feature point detection algorithm block diagram；
Fig. 4 is mechanical arm relative coordinate system schematic diagram of the present invention.
The technical characteristic that each appended drawing reference is referred in figure is as follows:
1, propeller；2, propeller motor；3, drone body；4, battery；5, mechanical arm pedestal；6, the first steering engine；7,
Second steering engine；8, third steering engine；9, mechanical arm connecting bracket；10, the 4th steering engine；11, the 5th steering engine；12, the 6th steering engine；13,
Camera；14, clamping jaw steering engine；15, clamping jaw.
Specific embodiment
Specific embodiments of the present invention will be further explained with reference to the accompanying drawing:
As shown in Figure 1, flight mechanical arm includes drone body 3 and mechanical arm, it is provided in the drone body 3
The propeller motor 2 that mechanical arm pedestal 5, several equally distributed propellers 1, control propeller rotate；The mechanical arm pedestal 5
An accommodation space is formed between drone body 3, the battery 4 is contained in the accommodation space, and the mechanical arm is solid
Dingan County is mounted in the lower surface of the mechanical arm pedestal 5, and multiple rotary joints are arranged in the mechanical arm, and each rotary joint is corresponding
A steering engine is arranged in one degree of freedom, each rotary joint, and the end of the mechanical arm is provided with clamping jaw 15 and camera shooting
First 13, it is additionally provided with mechanical arm connecting bracket 9 on the mechanical arm, is provided with control system in the drone body,
The control system controls the movement of the unmanned plane 3, mechanical arm and clamping jaw 15, the camera 13 and the control system electricity
Property connection, the mechanical arm adjusts the clamping jaw 15 of the mechanical arm tail end by the rotary joint to clamp object.Wherein,
The central axis of the central axis of the camera 13 and the clamping jaw 15, be in order to it is more acurrate, efficiently grab object.
The central axis of the clamping jaw 15 refers to the axis of the clamping direction of clamping jaw 15.The camera is colour imagery shot.
The mechanical arm has six-freedom degree, and each freedom degree constitutes institute by motor, train of reduction gears and encoder
State steering engine.Train of reduction gears is mounted on the output shaft of motor, and multiple pitch wheels are arranged in the train of reduction gears,
Go out on shaft gear the rotary shaft being arranged for connecting the steering wheel in steering engine in train of reduction gears.The steering engine further includes steering engine sheet
Body, steering engine pedestal, secondary steering wheel, main rudder disk, setting are circumferential along the steering gear body between the secondary steering wheel and the main rudder disk
The steering engine lateral wall of distribution, the steering gear body are arranged on the steering engine pedestal, and the pair steering wheel and main rudder disk are respectively set
In the outside of roof and bottom wall that the steering gear body is oppositely arranged.The main rudder disk and the setting of secondary steering wheel mutually to installation
Hole, the mounting hole are used for fixed conveyor connector, so that secondary steering wheel follows main rudder disk to move synchronously.
The steering engine includes the first steering engine 6, the second steering engine 7, third steering engine 8, the 4th steering engine 10, the 5th steering engine the 11, the 6th
Steering engine 12 and clamping jaw steering engine 14, the secondary steering wheel of first steering engine 6 are connected with the mechanical arm pedestal 5, first steering engine 6
Main rudder disk connect with the steering engine lateral wall of second steering engine 7 to drive the second rudder 7 rotation.The master of second steering engine 7
Steering wheel, which is connected to drive with one end affixed side for extending armed lever, extends armed lever rotation, the other end of the extension arm and described the
The steering engine pedestal of three steering engines 8 connects, and drives third steering engine 8 to rotate while extending armed lever rotation, the main rudder disk of the third rudder 8
It is connected respectively with one end of two connecting rods parallel to each other with secondary steering wheel, the other end of two connecting rods parallel to each other is fixed respectively
On the steering engine lateral wall of the 4th steering engine 10, realize that the main rudder disk of third steering engine 8 and secondary steering wheel drive the 4th by connecting rod
Steering engine 10 rotates.The main rudder disk of 4th steering engine 10 is connected to drive the 5th with the steering engine pedestal of the 5th steering engine 11
Steering engine 11 rotates, and the main rudder disk of the 5th steering engine 11 and secondary steering wheel are connected with one end of two connecting rods parallel to each other respectively,
The other end of two connecting rods parallel to each other is connected with the steering engine lateral wall of the 6th steering engine 12, realizes the 5th rudder by connecting rod
The main rudder disk of machine and secondary steering wheel drive the rotation of the 6th steering engine 12.The corresponding joint of each steering engine, so, the flight arm includes
First joint, second joint, third joint, the 4th joint, the 5th joint are with, the 6th joint and clamping jaw joint.
The present invention is mainly by the joint visual servo strategy two of the visual servo algorithm based on physical feature and multitask
It is grouped as, with regard to these two aspects expansion, expansion is discussed respectively below:
As shown in Fig. 2, the flight mechanical arm visual servo grasping means mainly include front end natural feature points extract and
Rear end visual servo resolves, and mainly includes the following steps:
Step 1: desired image of the shooting containing object, artificial frame take any rectangular area in object, record lower frame
Take region in the location of pixels of desired image angle point, the position for taking region in realtime graphic for finding frame detects in region
Characteristic point be denoted as desired character point；
Step 2: the camera arrests realtime graphic, the realtime graphic that the control system arrests the camera
Carry out characteristic point detection and matching；
Step 3: by the matching of realtime graphic and desired character point, the homography matrix transformation that can be obtained between image is closed
And then system utilizes the affine transformation of homography matrix, can find out corner location of the angle point of desired image in realtime graphic；
Step 4: it is poor to be made using the real-time corner location and desired corner location information that obtain, obtains the camera
Servo velocity；
Step: 5: by the velocity transformation relationship of robotics, the servo velocity of the camera can be transmitted to institute
It states unmanned plane and the mechanical arm carries out speed control.
The extraction of so-called front end natural feature points refers to, is not necessarily to any handmarking's object by the object of servo, only
Need to shoot the desired image containing object in advance, artificial frame takes any rectangular area in object, and record lower frame takes region
In the location of pixels of desired image angle point, the position for taking region in realtime graphic for finding frame, the feature detected in region
Point is denoted as desired character point；The image grabbed is caught by colour imagery shot, characteristic point detection and matching is then carried out, by scheming in real time
The matching with desired character point of picture, can obtain the homography matrix transformation relation between image, and then utilize homography matrix
Affine transformation, corner location of the angle point of desired image in realtime graphic can be found out.
Rear end visual servo is resolved using the real-time angle point based on the visual servo based on image, obtained using front end
It is poor that position and desired corner location information are made, and the front end as the visual servo based on image inputs, and may finally obtain phase
The servo velocity of machine.By the velocity transformation relationship of robotics, the servo velocity of camera can be transmitted to unmanned plane 3 and machine
Tool arm carries out speed control.
Features described above point uses ORB characteristic point, makes improvements, and algorithm flow chart is shown in Fig. 3, introduces Gaussian image gold word
Tower increases the scale invariability of characteristic point, due to characteristic point be unevenly distributed with, characteristic point is gathered on foreground object, is unfavorable for
Feature Points Matching, therefore introduce quad-tree structure gridding and extract characteristic point, guarantee that each of image is small by reducing threshold value
Grid spaces have characteristic point to be detected.
The joint visual servo strategy of multitask is as follows:
As shown in figure 4, flight mechanical arm define inertial coodinate system O, body coordinate system b, mechanical arm base coordinate system a,
Robot arm end effector coordinate system e, camera photocentre coordinate system c, object centre coordinate system t, expectation crawl position coordinate system
D, ob=[x y z]TAnd RbIndicate position and rotation of the unmanned plane coordinate system relative to inertial coodinate system,With Europe
The form at angle is drawn to indicate the direction of unmanned plane, q=[q1 q2 q3 q4 q5 q6]TIndicate the joint angle in each joint of mechanical arm.It connects down
It is briefly illustrated come the differential kinematics derivation to flight mechanical arm entirety, the speed etc. of mechanical arm tail end in flight mechanical arm
In the sum of the speed of speed and the generation of flight manipulator motion that unmanned plane movement generates:
ve=ve_a+ve_b (1)
Wherein veIndicate mechanical arm tail end final speed, ve_aIndicate the end of arm speed that manipulator motion generates, ve _bIndicate the end of arm speed that unmanned plane movement generates.
It is available according to the differential kinematics of mechanical arm:
Wherein J (q) indicates the geometry Jacobian matrix of mechanical arm,It is neat to indicate that mechanical arm positive kinematics solve
Rotational component in submatrix,Indicate the angular speed in each joint, J2Variable after indicating abbreviation.
Robot arm end effector speed goes under unmanned plane coordinate system and is expressed as:
WhereinIndicate the rotational component of unmanned plane pedestal relative mechanical arm end,Indicate stem-winder end with respect to nothing
The translational component of man-machine pedestal, S (*) indicate that vector turns antisymmetric matrix, vbIt is the movement velocity of unmanned plane, J1After indicating abbreviation
Variable.
Formula (2) and formula (3) are merged:
Multi-rotor unmanned aerial vehicle itself is a underactuated control system, controlled amounts be three degree of freedom linear velocity component and
Along the angular velocity of rotation of aircraft z-axis.Angular velocity of rotation passes through the inertial measuring unit of aircraft itself on remaining two axis
It obtains and is adjusted as the internal feedback ring of stabilized flight, therefore formula (4) can be write as:
ve=Jv=Jcovco+Jucvuc (5)
Wherein JucTo correspond to uncontrollable component in joint Jacobian matrix JColumn vector, JcoFor joint
Remove the Jacobian matrix of uncontrollable part in Jacobian matrix J,It is controllable
Component.
During practical flight, multi-rotor unmanned aerial vehicle remains steady state, i.e. pitch angle and roll angle is close
In zero, therefore uncontrollable part hardly impacts system, formula (5) can approximation write as:
ve=Jv=Jcovco+Jucvuc≈Jcovco (6)
Mechanical arm is the underface for being fixed on unmanned plane, and the movement of the two has the interference of power and torque to both sides, if
Visual servo crawl can be carried out in opereating specification using what unmanned plane hovered over mechanical arm, can significantly reduce it is mutual it
Between interference problem, but remote mechanical arm exceed working space when, mechanical arm can not grab, and be also impossible in actual task
Just flight mechanical arm, which is fallen in, to grab in range, therefore this patent needs the movement of visual servo being transmitted to unmanned plane,
The visual servo task of unmanned plane is done when remote.Camera is mounted on mechanical arm tail end, as unmanned plane servo leans on close-target
When object, object can progressively disengage the visual field of camera, if mechanical arm provides pitch angle and yaw angle real-time servo object,
It can guarantee object always in visual field center.Therefore this patent proposes that Speed allotment rate realizes that the joint vision of multitask is watched
Clothes control, setting speed allocation matrix are W ∈ R10*n, number of the n depending on depending on needing control amount, vu∈Rn*1For control amount
Vector, for practical problem vco
Extract corresponding vu, formula (6) can further be write as:
ve=JcoWvu=Juvu (7)
The visual servo process of flight mechanical arm is divided into remote and short distance.Mechanical arm and unmanned plane can when remote
To act on simultaneously, but to will lead to unmanned plane during flying unstable for the Large Amplitude Motion of mechanical arm and drone center of unmanned aerial vehicle offset, therefore
The visual servo that this patent uses unmanned plane to dominate, the linear velocity of the x, y, z axis direction of unmanned plane rotate along the z-axis direction
The speed in the 5th joint of angular speed and mechanical arm, the Speed allotment matrix in formula (7) can be write as:
It is closely the possible op space that unmanned plane enters mechanical arm, unmanned plane hovering, only manipulator motion, speed are divided
It can be write as with matrix:
Above to a kind of flight mechanical arm visual servo grasping means implementation based on physical feature provided by the present invention
Example is elaborated.Used herein a specific example illustrates the principle and implementation of the invention, above
The explanation of embodiment is merely used to help understand the core idea of the present invention.It should be pointed out that for the common skill of the art
For art personnel, without departing from the principles of the present invention, can with the present invention some improvement and modification can also be carried out, these change
It is also fallen within the protection scope of the claims of the present invention into modification.
Claims (7)
1. a kind of flight mechanical arm visual servo grasping means based on physical feature, it is characterised in that: flight mechanical arm includes
Drone body is provided with mechanical arm pedestal in the drone body, is provided with mechanical arm on the mechanical arm pedestal；
Multiple rotary joints are set in the mechanical arm, a steering engine is set in each rotary joint；At the end of the mechanical arm
End is provided with clamping jaw and camera；Control system is provided in the drone body, the control system controls the nothing
The movement of man-machine, mechanical arm and clamping jaw, the camera and the control system are electrically connected；The flight mechanical arm vision is watched
Grasping means is taken to include the following steps:
Step 1: desired image of the shooting containing object, artificial frame take any rectangular area in object, and record lower frame takes area
Location of pixels of the domain in desired image angle point, the position for taking region in realtime graphic for finding frame, the spy detected in region
Sign point is denoted as desired character point；
Step 2: the camera arrests realtime graphic, and the realtime graphic that the control system arrests the camera carries out
Characteristic point detection and matching；
Step 3: by the matching of realtime graphic and desired character point, the homography matrix transformation relation between image can be obtained,
And then the affine transformation for utilizing homography matrix, can find out corner location of the angle point of desired image in realtime graphic；
Step 4: it is poor to be made using the real-time corner location and desired corner location information that obtain, obtains the servo of the camera
Speed；
Step 5: by the velocity transformation relationship of robotics, the servo velocity of the camera can be transmitted to it is described nobody
Machine and the mechanical arm carry out speed control.
2. a kind of flight mechanical arm visual servo grasping means based on physical feature as described in claim 1, feature exist
In: the central axis of the camera is parallel with the central axis of the clamping jaw.
3. a kind of flight mechanical arm visual servo grasping means based on physical feature as described in claim 1, feature exist
In: the characteristic point uses ORB characteristic point.
4. a kind of flight mechanical arm visual servo grasping means based on physical feature as claimed in claim 3, feature exist
In: improved ORB characteristic point algorithm is as follows:
Step 2.1: input picture；
Step 2.2: Gaussian image pyramid；
Step 2.3: quadtree mesh divides；
Step 2.4: band direction FAST angle point grid；
Step 2.5: redundancy angle point is rejected；
Step 2.6: feature point description.
5. a kind of flight mechanical arm visual servo grasping means based on physical feature as described in claim 1, feature exist
In: the mechanical arm is capable of providing pitch angle and yaw angle.
6. a kind of flight mechanical arm visual servo grasping means based on physical feature as claimed in claim 5, feature exist
In: the mechanical arm includes the first joint, second joint, third joint, the 4th joint, the 5th joint and the 6th joint；It is described
The end of mechanical arm is the 6th joint, described mechanical when the distance of unmanned plane distance objective object is greater than threshold value, only controls nobody
The linear velocity of the x, y, z axis direction of machine, the speed in the 5th joint of the angular speed rotated along the z-axis direction and mechanical arm；When
When the distance of unmanned plane distance objective object is less than threshold value, the speed of mechanical arm is only controlled.
7. a kind of flight mechanical arm visual servo grasping means based on physical feature as claimed in claim 6, feature exist
In: the servo velocity of the camera is transmitted to the calculating process of the unmanned plane and the mechanical arm such as in the step 5
Under:
Inertial coodinate system O that flight mechanical arm defines, body coordinate system b, mechanical arm base coordinate system a, mechanical arm tail end execute
Device coordinate system e, camera photocentre coordinate system c, object centre coordinate system t, expectation crawl position coordinate system d, ob=[x y z]TWith
RbIndicate position and rotation of the unmanned plane coordinate system relative to inertial coodinate system,Indicated in the form of Eulerian angles without
Man-machine direction, q=[q1 q2 q3 q4 q5 q6]TIndicate the joint angle in each joint of mechanical arm, it is next whole to flight mechanical arm
The differential kinematics derivation of body is briefly illustrated that the speed of mechanical arm tail end is equal to unmanned plane movement and generates in flight mechanical arm
Speed and the sum of the speed that generates of flight manipulator motion:
ve=ve_a+ve_b (1)
Wherein veIndicate mechanical arm tail end final speed, ve_aIndicate the end of arm speed that manipulator motion generates, ve_bIt indicates
The end of arm speed that unmanned plane movement generates,
It is available according to the differential kinematics of mechanical arm:
Wherein J (q) indicates the geometry Jacobian matrix of mechanical arm,Indicate the homogeneous square that mechanical arm positive kinematics solve
Rotational component in battle array,Indicate the angular speed in each joint, J2Variable after indicating abbreviation,
Robot arm end effector speed goes under unmanned plane coordinate system and is expressed as:
WhereinIndicate the rotational component of unmanned plane pedestal relative mechanical arm end,Indicate stem-winder end with respect to unmanned plane
The translational component of pedestal, S (*) indicate that vector turns antisymmetric matrix, vbIt is the movement velocity of unmanned plane, J1Change after indicating abbreviation
Amount,
Formula (2) and formula (3) are merged:
Multi-rotor unmanned aerial vehicle itself is a underactuated control system, controlled amounts be three degree of freedom linear velocity component and along
The angular velocity of rotation of aircraft z-axis is left angular velocity of rotation on two axis and is obtained by the inertial measuring unit of aircraft itself
Internal feedback ring as stabilized flight is adjusted, therefore formula (4) can be write as:
ve=Jv=Jcovco+Jucvuc (5)
Wherein JucTo correspond to uncontrollable component in joint Jacobian matrix JColumn vector, JcoIt is refined gram of joint
Than the Jacobian matrix for removing uncontrollable part in matrix J,For controllable component,
During practical flight, multi-rotor unmanned aerial vehicle remains steady state, i.e. pitch angle and roll angle close to zero,
Therefore uncontrollable part hardly system is impacted, formula (5) can approximation write as:
ve=Jv=Jcovco+Jucvuc≈Jcovco (6)
Setting speed allocation matrix is W ∈ R10*n, number of the n depending on depending on needing control amount, vu∈Rn*1For control amount
Vector, for practical problem vco
Extract corresponding vu, formula (6) can further be write as:
ve=JcoWvu=Juvu (7)
When the distance of unmanned plane distance objective object is greater than threshold value, the Speed allotment matrix in formula (7) can be write as:
When the distance of unmanned plane distance objective object is less than threshold value, the Speed allotment matrix in formula (7) can be write as:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910241810.9A CN109895099B (en) | 2019-03-28 | 2019-03-28 | Flying mechanical arm visual servo grabbing method based on natural features |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910241810.9A CN109895099B (en) | 2019-03-28 | 2019-03-28 | Flying mechanical arm visual servo grabbing method based on natural features |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109895099A true CN109895099A (en) | 2019-06-18 |
| CN109895099B CN109895099B (en) | 2020-10-02 |
Family
ID=66953908
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910241810.9A Active CN109895099B (en) | 2019-03-28 | 2019-03-28 | Flying mechanical arm visual servo grabbing method based on natural features |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109895099B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110286688A (en) * | 2019-06-19 | 2019-09-27 | 广东工业大学 | A kind of underwater quadrotor drone control method with mechanical arm |
| CN110427043A (en) * | 2019-09-04 | 2019-11-08 | 福州大学 | Pose Control device design method based on operation flying robot's centre-of gravity shift |
| CN111015673A (en) * | 2020-01-02 | 2020-04-17 | 福州大学 | Four-degree-of-freedom mechanical arm teleoperation system and method for operation type flying robot |
| CN111360840A (en) * | 2020-04-28 | 2020-07-03 | 沈阳优诚自动化工程有限公司 | Mechanical arm sorting method based on visual algorithm |
| CN111923049A (en) * | 2020-08-21 | 2020-11-13 | 福州大学 | Flight mechanical arm visual servo and multi-task control method based on spherical model |
| CN112589787A (en) * | 2020-12-02 | 2021-04-02 | 上海纽钛测控技术有限公司 | Visual positioning and hand-eye calibration method for loading and unloading samples of mechanical arm of feeding turntable |
| CN114291264A (en) * | 2022-02-14 | 2022-04-08 | 天津七六四通信导航技术有限公司 | Unmanned aerial vehicle's first aid device |
| CN114536327A (en) * | 2022-01-24 | 2022-05-27 | 四川广目科技有限公司 | Intelligent industrial mechanical arm driving system based on ROS system |
| CN114842056A (en) * | 2022-04-19 | 2022-08-02 | 深圳鳍源科技有限公司 | Multi-machine-position first machine visual angle following method, system, device and equipment |
| CN114862848A (en) * | 2022-07-05 | 2022-08-05 | 江苏顺联工程建设有限公司 | Intelligent control method of hoisting equipment for municipal construction |
| CN115520376A (en) * | 2022-09-27 | 2022-12-27 | 哈尔滨工业大学 | Air-ground dual-purpose mobile operation platform based on vector rotor wing and pose control system and method thereof |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106570820A (en) * | 2016-10-18 | 2017-04-19 | 浙江工业大学 | Monocular visual 3D feature extraction method based on four-rotor unmanned aerial vehicle (UAV) |
| CN106985159A (en) * | 2017-05-10 | 2017-07-28 | 哈尔滨工业大学深圳研究生院 | A kind of flight mechanical arm with flexible crawl device |
| CN107139178A (en) * | 2017-05-10 | 2017-09-08 | 哈尔滨工业大学深圳研究生院 | A kind of grasping means of unmanned plane and its view-based access control model |
| US20170259920A1 (en) * | 2016-03-10 | 2017-09-14 | International Business Machines Corporation | Automatic painting system with drone, user interface and computer vision |
| CN107627303A (en) * | 2017-09-22 | 2018-01-26 | 哈尔滨工程大学 | A kind of PD SMC control methods based on vision servo system of the eye in structure on hand |
| CN107902089A (en) * | 2017-12-14 | 2018-04-13 | 郑州启硕电子科技有限公司 | A kind of unmanned plane of scalable crawl |
| CN108015764A (en) * | 2017-11-20 | 2018-05-11 | 中国运载火箭技术研究院 | A kind of zero priori target acquistion method of space based on the fusion of multi-source visual information |
| CN108170160A (en) * | 2017-12-21 | 2018-06-15 | 中山大学 | It is a kind of to utilize monocular vision and the autonomous grasping means of airborne sensor rotor wing unmanned aerial vehicle |
| US20180186570A1 (en) * | 2015-03-24 | 2018-07-05 | Joseph Porat | System and method for overhead warehousing |
| CN108453738A (en) * | 2018-03-30 | 2018-08-28 | 东南大学 | A kind of quadrotor based on Opencv image procossings independently captures the control method of operation in the air |
| CN208181424U (en) * | 2018-04-25 | 2018-12-04 | 浙江工业大学 | The unmanned plane Full-automatic grasping device of view-based access control model identifying system |
| CN109398688A (en) * | 2018-11-16 | 2019-03-01 | 湖南大学 | A kind of rotor flying double mechanical arms target positioning grasping system and method |
-
2019
- 2019-03-28 CN CN201910241810.9A patent/CN109895099B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180186570A1 (en) * | 2015-03-24 | 2018-07-05 | Joseph Porat | System and method for overhead warehousing |
| US20170259920A1 (en) * | 2016-03-10 | 2017-09-14 | International Business Machines Corporation | Automatic painting system with drone, user interface and computer vision |
| CN106570820A (en) * | 2016-10-18 | 2017-04-19 | 浙江工业大学 | Monocular visual 3D feature extraction method based on four-rotor unmanned aerial vehicle (UAV) |
| CN106985159A (en) * | 2017-05-10 | 2017-07-28 | 哈尔滨工业大学深圳研究生院 | A kind of flight mechanical arm with flexible crawl device |
| CN107139178A (en) * | 2017-05-10 | 2017-09-08 | 哈尔滨工业大学深圳研究生院 | A kind of grasping means of unmanned plane and its view-based access control model |
| CN107627303A (en) * | 2017-09-22 | 2018-01-26 | 哈尔滨工程大学 | A kind of PD SMC control methods based on vision servo system of the eye in structure on hand |
| CN108015764A (en) * | 2017-11-20 | 2018-05-11 | 中国运载火箭技术研究院 | A kind of zero priori target acquistion method of space based on the fusion of multi-source visual information |
| CN107902089A (en) * | 2017-12-14 | 2018-04-13 | 郑州启硕电子科技有限公司 | A kind of unmanned plane of scalable crawl |
| CN108170160A (en) * | 2017-12-21 | 2018-06-15 | 中山大学 | It is a kind of to utilize monocular vision and the autonomous grasping means of airborne sensor rotor wing unmanned aerial vehicle |
| CN108453738A (en) * | 2018-03-30 | 2018-08-28 | 东南大学 | A kind of quadrotor based on Opencv image procossings independently captures the control method of operation in the air |
| CN208181424U (en) * | 2018-04-25 | 2018-12-04 | 浙江工业大学 | The unmanned plane Full-automatic grasping device of view-based access control model identifying system |
| CN109398688A (en) * | 2018-11-16 | 2019-03-01 | 湖南大学 | A kind of rotor flying double mechanical arms target positioning grasping system and method |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110286688B (en) * | 2019-06-19 | 2021-03-16 | 广东工业大学 | Control method for underwater four-rotor unmanned aerial vehicle with mechanical arm |
| CN110286688A (en) * | 2019-06-19 | 2019-09-27 | 广东工业大学 | A kind of underwater quadrotor drone control method with mechanical arm |
| CN110427043A (en) * | 2019-09-04 | 2019-11-08 | 福州大学 | Pose Control device design method based on operation flying robot's centre-of gravity shift |
| CN110427043B (en) * | 2019-09-04 | 2021-09-28 | 福州大学 | Pose controller design method based on gravity center offset of operation flying robot |
| CN111015673A (en) * | 2020-01-02 | 2020-04-17 | 福州大学 | Four-degree-of-freedom mechanical arm teleoperation system and method for operation type flying robot |
| CN111360840A (en) * | 2020-04-28 | 2020-07-03 | 沈阳优诚自动化工程有限公司 | Mechanical arm sorting method based on visual algorithm |
| CN111923049B (en) * | 2020-08-21 | 2023-11-03 | 福州大学 | Visual servo and multitasking control method for flying mechanical arm based on spherical model |
| CN111923049A (en) * | 2020-08-21 | 2020-11-13 | 福州大学 | Flight mechanical arm visual servo and multi-task control method based on spherical model |
| CN112589787A (en) * | 2020-12-02 | 2021-04-02 | 上海纽钛测控技术有限公司 | Visual positioning and hand-eye calibration method for loading and unloading samples of mechanical arm of feeding turntable |
| CN114536327A (en) * | 2022-01-24 | 2022-05-27 | 四川广目科技有限公司 | Intelligent industrial mechanical arm driving system based on ROS system |
| CN114291264A (en) * | 2022-02-14 | 2022-04-08 | 天津七六四通信导航技术有限公司 | Unmanned aerial vehicle's first aid device |
| CN114842056A (en) * | 2022-04-19 | 2022-08-02 | 深圳鳍源科技有限公司 | Multi-machine-position first machine visual angle following method, system, device and equipment |
| CN114862848A (en) * | 2022-07-05 | 2022-08-05 | 江苏顺联工程建设有限公司 | Intelligent control method of hoisting equipment for municipal construction |
| CN115520376A (en) * | 2022-09-27 | 2022-12-27 | 哈尔滨工业大学 | Air-ground dual-purpose mobile operation platform based on vector rotor wing and pose control system and method thereof |
| CN115520376B (en) * | 2022-09-27 | 2023-08-25 | 哈尔滨工业大学 | Space-ground-based dual-purpose mobile operation platform and pose control system and control method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109895099B (en) | 2020-10-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109895099A (en) | A kind of flight mechanical arm visual servo grasping means based on physical feature | |
| CN108453738B (en) | Control method for four-rotor aircraft aerial autonomous grabbing operation based on Opencv image processing | |
| CN107014380B (en) | Combined navigation method based on visual navigation and inertial navigation of aircraft | |
| CN105022401B (en) | Many four rotor wing unmanned aerial vehicles collaboration SLAM methods of view-based access control model | |
| CN109079799B (en) | Robot perception control system and control method based on bionics | |
| CN108563235B (en) | Multi-rotor unmanned aerial vehicle, method, device and equipment for grabbing target object | |
| CN108248845A (en) | A kind of rotor flying mechanical arm system and algorithm based on dynamic center of gravity compensation | |
| CN101566476B (en) | Scene matching semi-physical simulation system based on mechanical arm with six degree of freedom | |
| CN102190081B (en) | Vision-based fixed point robust control method for airship | |
| CN103365297A (en) | Optical flow-based four-rotor unmanned aerial vehicle flight control method | |
| CN105717933A (en) | Unmanned aerial vehicle and unmanned aerial vehicle anti-collision method | |
| CN109178345B (en) | Cloud deck pointing and star attitude cooperative control method for aerial moving target tracking | |
| Mills et al. | Vision based control for fixed wing UAVs inspecting locally linear infrastructure using skid-to-turn maneuvers | |
| CN108945536B (en) | Rendezvous and docking experimental platform based on rotor craft | |
| Luo et al. | Natural feature-based visual servoing for grasping target with an aerial manipulator | |
| CN108312137A (en) | UAV Landing docking mechanism based on multi-joint mechanical arm | |
| Zhang et al. | Recent advances on robot visual servo control methods | |
| Amidi et al. | Research on an autonomous vision-guided helicopter | |
| Razinkova et al. | Tracking a moving ground object using quadcopter UAV in a presence of noise | |
| Xiao-Hong et al. | UAV's automatic landing in all weather based on the cooperative object and computer vision | |
| Du et al. | Pose estimation of large non-cooperative spacecraft based on extended PnP model | |
| CN108170160A (en) | It is a kind of to utilize monocular vision and the autonomous grasping means of airborne sensor rotor wing unmanned aerial vehicle | |
| CN206913156U (en) | A kind of unmanned plane | |
| CN110949684A (en) | Low-speed unmanned aerial vehicle air-based recovery system based on mechanical smart gripper system | |
| Lee et al. | Landing Site Inspection and Autonomous Pose Correction for Unmanned Aerial Vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |