CN115521864A

CN115521864A - Remote operation's force feedback self-adaptation micromanipulator

Info

Publication number: CN115521864A
Application number: CN202211508067.7A
Authority: CN
Inventors: 陈方鑫; 韦俊杰; 毕海
Original assignee: Ji Hua Laboratory
Current assignee: Ji Hua Laboratory
Priority date: 2022-11-29
Filing date: 2022-11-29
Publication date: 2022-12-27

Abstract

The invention relates to the technical field of micromanipulation, and discloses a force feedback self-adaptive micromanipulator operated remotely. The remotely operated force feedback adaptive micromanipulator comprises: the cell stage is used for accommodating cells needing to be operated; the actuating mechanism comprises two mechanical arms, puncture needles and clamping devices which are respectively arranged at the tail ends of the two mechanical arms, the mechanical arms have the degrees of freedom in the three directions of an x axis, a y axis and a z axis, and the two mechanical arms are respectively arranged at two sides of the cell stage; the visual module is used for acquiring the position information of the cells, the puncture needle and the holder, and is arranged on one side of the cell stage; and the operation master hand is used for controlling the action of the actuating mechanism. The operator can remotely control the micromanipulator to perform cell operation through the operation master hand, and the operation master hand is provided with a resistance feedback function which enables the operator to obtain more real hand feeling, so that the cell operation quality is improved.

Description

Remote operation's force feedback self-adaptation micromanipulator

Technical Field

The invention relates to the technical field of microscopic equipment, in particular to a force feedback self-adaptive microscopic operation instrument for remote operation.

Background

The fields of life sciences and biomedicine require a large number of cell injection procedures. The quality of the cell injection procedure can seriously affect the consistency and reproducibility of the experimental results. The operation is only observed by human eyes to judge whether the operation is successful, force feedback is lacked, and the operation difficulty is increased. Force feedback provides the operator with force information except visual information, and the degree of cell clamping and puncturing is judged through the force feedback information, so that the operation force is changed autonomously, and the cell operation quality is favorably provided.

In the prior art, the Chinese patent with the application number of 201610138090 and the name of 'a biological laboratory micromanipulator' discloses a micromanipulator for biological cell experiments. The scheme of the Chinese invention patent with the application number of 202010728717.3 and the name of a micro-nano execution system for realizing cell operation realizes the puncture action of cells by utilizing piezoelectric ceramics and a stroke amplification mechanism, and increases the movement precision through the feedback of a sensor. In addition to the two cases, the current patent and scientific literature of the invention aim to improve the precision and automation of cell micromanipulation, and the proposed method indeed exerts a positive gain effect on the above-mentioned aim. However, in biological experiments, there are problems that cell manipulation cannot be performed remotely, manipulation difficulty is high, and skill requirements on operators are high.

Disclosure of Invention

The invention mainly aims to provide a remote-operated force feedback self-adaptive micromanipulator, aiming at solving the technical problems of high difficulty in cell operation and high requirement on the skill of an operator.

To achieve the above object, the present invention provides a remotely operated force feedback adaptive micromanipulator, comprising: the cell stage is used for accommodating cells needing to be operated; the actuating mechanism comprises two mechanical arms, puncture needles and clamping devices which are respectively arranged at the tail ends of the two mechanical arms, the mechanical arms have the degrees of freedom in the three directions of an x axis, a y axis and a z axis, and the two mechanical arms are respectively arranged at two sides of the cell stage; the visual module is used for acquiring real-time images of the cells, the puncture needle and the holder, and is arranged on one side of the cell stage; the operation main hand is used for controlling the action of the execution mechanism, feeding back the puncture force when the puncture needle punctures the cells and feeding back the clamping force when the clamp holder clamps the cells, and the operation main hand is connected with the execution mechanism through the operation main hand and the execution mechanism through remote signals.

Optionally, the vision module includes light source, CCD camera and microscope, the light source set up in cell objective table top, the microscope set up in the below of cell objective table, the CCD camera set up in the microscope is kept away from on the one side of cell objective table, the CCD camera is used for shooting the cell and is in take place the real-time image of deformation under the actuating mechanism effect.

Optionally, a remote control module is arranged in the force feedback adaptive micro-manipulator of the remote operation, the remote control module is in communication connection with the operation master hand, the remote control module is in communication connection with the vision module, and the remote control module is in communication connection with the execution mechanism.

Optionally, the operation owner hand includes handle, control button and force feedback subassembly, control button set up in on the handle, the force feedback subassembly including set up in the inside torque motor of handle, with the hooke's hinge that torque motor connects and with the rocker that hooke's hinge is connected, the rocker stretches out the handle surface, the operation owner hand with remote control module communication is connected, torque motor is used for output feedback power when actuating mechanism operates the cell.

Optionally, the clamping force is calculated by the remotely operated force feedback adaptive micromanipulator through a first visual algorithm, and an expression of the first visual algorithm is as follows:

；

wherein the content of the first and second substances,

in order to provide a clamping force, the clamping device,

for the rigidity of the cell clamping deformation,

for cell clamping deformation.

Optionally, the puncture force is calculated by the remotely operated force feedback adaptive micromanipulator through a second visual algorithm, and an expression of the second visual algorithm is as follows:

；

wherein, the first and the second end of the pipe are connected with each other,

in order to provide the force for the puncture,

the rigidity of the deformation is changed for the cell puncture,

and (5) carrying out cell puncture deformation.

Optionally, the cell stage includes a culture dish, an x-axis movement assembly and a y-axis movement assembly, the y-axis movement assembly is disposed on the x-axis movement assembly, and the culture dish is disposed on the y-axis movement assembly.

Optionally, the cell stage further comprises a rotation motion assembly, the rotation motion assembly is disposed on the y-axis motion assembly, and the culture dish is disposed on the rotation motion assembly.

Optionally, be equipped with electronic revolving stage on the arm, electronic revolving stage uses the x axle to rotate as the center, the pjncture needle with the holder sets up respectively in one on the electronic revolving stage.

Optionally, the gripper end is provided with a pair of jaws.

In the technical scheme provided by the invention, an operator can remotely control the actuating mechanism by operating the main hand and inject or clamp the cells on the cell stage by the puncture needle or the clamp holder. The real-time images acquired by the vision module comprise images of the change of the shape and the position of the cell under the action of the holder or the puncture needle, and images of the change of the position and the change of the working state of the holder and the puncture needle in the cell stage. The vision module acquires a real-time image of deformation of the operated cell, and calculates acting forces applied when the holder and the puncture needle are respectively contacted with the operated cell through an algorithm in an internal processing module of the remote-operated force feedback self-adaptive micromanipulator, wherein the holding force is holding resistance when the holder holds the cell, and the puncture force is cell piercing resistance when the puncture needle injects the cell; then the vision module transmits a corresponding force signal to the operation master hand, and the operation master hand provides corresponding operation resistance, so that an operator obtains the sense of reality of cell clamping and puncturing operation, and further adjusts the operation force and direction to realize high-quality cell operation.

Drawings

One or more embodiments are illustrated in corresponding drawings which are not intended to be limiting, in which elements having the same reference number designation may be referred to as similar elements throughout the drawings, unless otherwise specified, and in which the drawings are not to scale.

FIG. 1 is a schematic structural view of one embodiment of a remotely operated force feedback adaptive micromanipulator of the present invention;

FIG. 2 is a schematic structural diagram of one embodiment of a master manipulator of the remotely operated force feedback adaptive micromanipulator of the present invention;

FIG. 3 is a schematic diagram of the construction of one embodiment of the force feedback assembly of the remotely operated force feedback adaptive micromanipulator of the present invention;

FIG. 4 is an enlarged view of a portion of FIG. 1 at A;

FIG. 5 is a partial enlarged view of the portion B in FIG. 4;

FIG. 6 is a schematic diagram of the system architecture of the remotely operated force feedback adaptive micromanipulator of the present invention.

Wherein, 1, force feedback adaptive micromanipulator of remote operation; 2. a cell stage; 21. a culture dish; 22. an x-axis motion assembly; 23. a y-axis motion assembly; 24. a rotational motion assembly; 241. a rotating electric machine; 242. a first tooth-shaped member; 243. a second toothed member; 3. an actuator; 31. a mechanical arm; 32. a holder; 321. a clamping jaw; 322. cells to be manipulated; 33. puncturing needle; 34. an electric rotating table; 4. a vision module; 41. a light source; 42. a microscope; 43. a CCD camera; 5. operating the master hand; 51. a handle; 52. a control button; 53. a force feedback assembly; 531. a torque motor; 532. hooke's joint; 533. a rocker.

Detailed Description

In order to facilitate an understanding of the invention, the invention is described in more detail below with reference to the accompanying drawings and specific examples. It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may be present. The terms "vertical," "horizontal," "left," "right," "inner," "outer," and the like as used herein are for descriptive purposes only. In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating relative importance or as implicitly indicating the number of technical features indicated. Thus, unless otherwise specified, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; "plurality" means two or more. The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that one or more other features, integers, steps, operations, elements, components, and/or combinations thereof may be present or added.

Furthermore, unless expressly stated or limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly and may include, for example, fixed connections, removable connections, and integral connections; can be mechanically or electrically connected; either directly or indirectly through intervening media, or through both elements. All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

Furthermore, the technical features mentioned in the different embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.

As shown in fig. 1 and 6, the present invention provides a remotely operated force feedback adaptive micromanipulator 1, the remotely operated force feedback adaptive micromanipulator 1 including: a cell stage 2 for accommodating cells to be manipulated; the actuating mechanism 3 comprises two mechanical arms 31, puncture needles 33 and grippers 32 which are respectively arranged at the tail ends of the two mechanical arms 31, the mechanical arms 31 have the freedom degrees in the three directions of an x axis, a y axis and a z axis, and the two mechanical arms 31 are respectively arranged at two sides of the cell stage 2; the vision module 4 is used for acquiring the position information of the cells, the puncture needle 33 and the holder 32, and the vision module 4 is arranged on one side of the cell stage 2; the operation main hand 5 is used for controlling the action of the actuating mechanism 3 and controlling the action of the actuating mechanism 3, and feeds back the puncture force when the puncture needle 33 punctures the cell and the clamping force when the feedback clamper 32 clamps the cell, and the operation main hand 5 is in remote signal connection with the actuating mechanism 3.

In the technical scheme provided by the invention, an operator can remotely control the actuating mechanism 3 by operating the main hand 5 and inject or clamp the cell on the cell stage 2 through the puncture needle 33 or the clamp holder 32. The real-time images acquired by the vision module 4 include images of the change of the shape and position of the cell under the action of the holder 32 or the puncture needle 33, and images of the change of the position and the change of the working state of the holder 32 and the puncture needle 33 in the cell stage 2. The vision module 4 acquires a real-time image of deformation of the operated cell, and calculates acting forces applied when the holder 32 and the puncture needle 33 are respectively contacted with the operated cell through an algorithm in an internal processing module of the remote-operated force feedback adaptive micromanipulator 1, wherein the holding force is holding resistance when the holder 32 holds the cell, and the puncture force is cell piercing resistance when the puncture needle 33 injects the cell; then the vision module 4 transmits a corresponding force signal to the operation main hand 5, and the operation main hand 5 provides a corresponding operation resistance, so that an operator obtains the sense of reality of cell clamping and puncturing operation, and further the operation force and direction are adjusted, and high-quality cell operation is realized.

As shown in fig. 1, in the present embodiment, the vision module 4 includes a light source 41, a CCD camera 43 and a microscope 42, which are disposed above the cell stage 2, the light source 41 is disposed above the cell stage 2, the microscope 42 is disposed below the cell stage 2, the CCD camera 43 is disposed on a surface of the microscope 42 away from the cell stage 2, and the CCD camera 43 is used for capturing an image of the cell deformed by the actuator 3. The cell stage 2 is provided with a culture dish 21, the culture dish 21 contains cell culture fluid, the whole culture dish 21 has light transmittance, and the CCD camera 43 arranged under the stage can shoot cell images in the culture dish 21. The microscope 42 is arranged on the lens of the CCD camera 43, and is matched with the light source 41 above the cell objective table 2 to form the whole visual module 4, the visual module 4 further comprises a display screen electrically connected with the remote-operated force feedback self-adaptive micromanipulator 1, and an operator can clearly observe the cell posture in the culture dish 21 through the display screen.

As shown in fig. 1 and fig. 6, in this embodiment, a remote control module is disposed in the force feedback adaptive micromanipulator 1, the remote control module is in communication connection with the operation master 5, the remote control module is in communication connection with the vision module 4, and the remote control module is in communication connection with the actuator 3. The remote control module is a 5G communication module, and the operation master hand 5 can be in communication connection with the execution mechanism 3 or the vision module 4 through the remote control module respectively, so that the remote operation of an operator is realized. For example, the operator operates the force feedback adaptive operation master 5 in the foshan, and the operation signal is transmitted to the remotely operated force feedback adaptive micromanipulator 1 in suzhou through the remote control module, at this time, the remotely operated force feedback adaptive micromanipulator 1 in suzhou performs corresponding operation according to the operator instruction in the foshan. When a certain cell operation is very complicated, human errors exist in the operation of different operators, and the repeatability of biological experiments is influenced by the human errors, the remote operation function provided by the invention can eliminate the human errors, and the influence of the human factors can be eliminated in a simple mode.

As shown in fig. 1 to 3, in the present embodiment, the operating master 5 includes a handle 51, a control button 52 and a force feedback assembly 53, the control button 52 is disposed on the handle 51, the force feedback assembly 53 includes a torque motor 531 disposed inside the handle 51, a hooke 532 connected to the torque motor 531 and a rocker 533 connected to the hooke 532, the rocker 533 extends out of the surface of the handle 51, the operating master 5 is connected to the remote control module in a communication manner, and the torque motor 531 is used for outputting a feedback force when the actuator 3 operates the cell. The lower end of the rocker 533 realizes two degrees of freedom movement through a hooke hinge 532, wherein the horizontal movement hinge is connected with a torque motor 531 to realize force feedback. Through the force signal received by the remote control module, the torque motor 531 is controlled to increase the resistance of the rocker 533, so that the operator can sense the feedback force. Two rockers 533 are provided in the force feedback assembly 53, wherein each of the two rockers 533 controls the movement of one of the robot arms 31. When the rocker 533 is pulled horizontally, the corresponding controlled mechanical arm 31 moves in the X direction in the visual field, and when the rocker 533 is pulled up and down, the mechanical arm 31 moves in the Y direction in the visual field. The control button 52 is used to switch the operation object, and when the control button 52 is pressed, the control object is changed from the robot arm 31 to the gripper 32 or the puncture needle 33, thereby controlling the movement of the robot arm 31 to be converted into the opening and closing movement of the gripper 32 or the injection of the injection needle.

As shown in fig. 1 and fig. 6, in this embodiment, the vision module 4 obtains a real-time image of the cell held by the holder 32, the force feedback adaptive microscopy generates a deformed image according to the cell, the holding force is calculated by a first vision algorithm and a corresponding force signal is sent, the first vision algorithm is an algorithm including cell holding stiffness and cell holding shape recognition, and the expression of the first vision algorithm is:

；

wherein the content of the first and second substances,

in order to provide a clamping force, the clamping device,

for the rigidity of the cell clamping deformation,

for cell clamping deformation. After the force signal is transmitted back to the operating main hand 5 through the remote control module, the operating main hand 5 controls the torque motor 531 to increase the resistance of the rocker 533 according to the force signal, so that the operator can sense the clamping force.

As shown in fig. 1 and fig. 6, in this embodiment, the mechanical arm 31 drives the puncture needle 33 to achieve cell puncture, the vision module 4 obtains a real-time image of the cell injected by the puncture needle 33, the force feedback adaptive microscopy instrument calculates a puncture force according to an image of cell deformation, and sends a corresponding force signal through a second vision algorithm, the second vision algorithm is an algorithm including cell puncture stiffness and cell puncture shape identification, and the expression is:

；

wherein the content of the first and second substances,

in order to provide the force for the puncture,

the rigidity of the cell puncture deformation is improved,

and (5) carrying out cell puncture deformation. The force signal is transmitted back to the operation main hand 5 through the remote control module, and the operation main hand 5 controls the torque motor 531 to increase the resistance of the rocker 533 according to the force signal, so that the operator can sense the puncture force.

As shown in fig. 1 to 3, in the present embodiment, the cell stage 2 includes a culture dish 21, an x-axis movement assembly 22 and a y-axis movement assembly 23, the y-axis movement assembly 23 is disposed on the x-axis movement assembly 22, and the culture dish 21 is disposed on the y-axis movement assembly 23. The x-axis motion assembly 22 and the y-axis motion assembly 23 are linear motion modules composed of a servo motor, a ball screw, a coupler and a screw sliding table. The culture dish 21 is arranged on the y-axis movement assembly 23, and the movement in the x-axis direction and the y-axis direction is realized under the driving of the x-axis movement assembly 22 and the y-axis movement assembly 23, so that the actuator 3 can move to any position in the culture dish 21 more conveniently.

As shown in fig. 1 to 4, in the present embodiment, the cell stage 2 further includes a rotation motion assembly 24, the rotation motion assembly 24 is disposed on the y-axis motion assembly 23, and the culture dish 21 is disposed on the rotation motion assembly 24. The rotating motion assembly 24 includes a rotating motor 241, a first tooth member 242, and a second tooth member 243 engaged with the first tooth member 242, and the culture dish 21 is disposed on the second tooth member 243. The rotating motor 241 is energized to rotate the first tooth 242, so as to rotate the culture dish 21 on the second tooth 243, and the operator can adjust to a proper pose to operate and observe the cells in the culture dish 21, thereby ensuring that no interference occurs between the puncture needle 33 and the holder 32 on the actuator 3 and the corresponding mechanical arm 31.

As shown in fig. 1 to 3, in the present embodiment, the robot arm 31 is provided with an electric turntable 34, the electric turntable 34 rotates about the x-axis, and the puncture needle 33 and the clamper 32 are provided on one electric turntable 34. The electric rotating platform 34 comprises a driving motor and a rotating sliding platform driven by the driving motor, the rotating sliding platform can be rotatably arranged on the mechanical arm 31, and the puncture needle 33 or the holder 32 is arranged on the electric rotating platform 34 and is driven by the electric rotating platform 34 to perform rotary displacement. The provision of the electric rotating table 34 increases the degree of freedom of the entire actuator 3, and enables the gripper 32, the puncture needle 33, or other cell manipulation device to manipulate the cell in an optimum position and posture.

As shown in fig. 4-5, in one embodiment, the gripper 32 is provided with a pair of jaws 321 at the distal end. The holder 32 can be a micromanipulation instrument modified by a pneumatic microneedle plus a mechanical gripper 321, the distal gripper 321 being driven by a pneumatic element to grip and release the cell 322 to be manipulated. Compared with the direct adsorption cell grasping method, the clamping jaw 321 of the invention increases the contact area of the clamping device 32 and the cell, and reduces the probability of surface damage of the cell in the extraction process.

The above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remotely operated force feedback adaptive micromanipulator, comprising:

the cell stage is used for accommodating cells needing to be operated;

the executing mechanism comprises two mechanical arms, puncture needles and clamping devices which are respectively arranged at the tail ends of the two mechanical arms, the mechanical arms have the freedom degrees in the three directions of an x axis, a y axis and a z axis, and the two mechanical arms are respectively arranged at two sides of the cell stage;

the visual module is used for acquiring real-time images of the cells, the puncture needle and the holder, and is arranged on one side of the cell stage;

and the operation master hand is used for controlling the action of the actuating mechanism, feeding back the puncture force when the puncture needle punctures the cells and feeding back the clamping force when the clamper clamps the cells, and is in remote signal connection with the actuating mechanism.

2. The remotely operated force feedback adaptive micromanipulator of claim 1, wherein the vision module comprises a light source, a CCD camera and a microscope, the light source is disposed above the cell stage, the microscope is disposed below the cell stage, the CCD camera is disposed on a surface of the microscope away from the cell stage, and the CCD camera is used for capturing real-time images of the cells deformed by the actuator.

3. The remotely operated force feedback adaptive micromanipulator of claim 1 or 2, wherein a remote control module is provided within the remotely operated force feedback adaptive micromanipulator, the remote control module being communicatively coupled to the manipulation master, the remote control module being communicatively coupled to the vision module, the remote control module being communicatively coupled to the actuator.

4. The remote-operated force feedback adaptive micromanipulator according to claim 3, wherein the operating master hand comprises a handle, a control button and a force feedback component, the control button is arranged on the handle, the force feedback component comprises a torque motor arranged in the handle, a hook joint connected with the torque motor and a rocker connected with the hook joint, the rocker extends out of the surface of the handle, the operating master hand is in communication connection with the remote control module, and the torque motor is used for outputting feedback force when the actuating mechanism operates the cells.

5. The remotely operated force feedback adaptive micromanipulator of claim 4, wherein the remotely operated force feedback adaptive micromanipulator calculates the clamping force by a first visual algorithm, the expression of which is:

；

wherein the content of the first and second substances,

in order to provide a holding force, the holding force,

for the rigidity of the cell clamping deformation,

for cell clamping deformation.

6. The remotely operated force feedback adaptive micromanipulator of claim 4, wherein the remotely operated force feedback adaptive micromanipulator calculates the penetration force by a second visual algorithm, the expression of which is:

；

in order to provide the force for the puncture,

the rigidity of the cell puncture deformation is improved,

deforming for cell puncture.

7. The remotely operated force feedback adaptive micromanipulator of claim 1, wherein the cell stage comprises a culture dish, an x-axis motion assembly and a y-axis motion assembly, the y-axis motion assembly being disposed on the x-axis motion assembly, the culture dish being disposed on the y-axis motion assembly.

8. The remotely operated force feedback adaptive micromanipulator of claim 7, wherein the cell stage further comprises a rotational motion assembly, the rotational motion assembly being disposed on the y-axis motion assembly, the culture dish being disposed on the rotational motion assembly.

9. The remotely operated force feedback adaptive micromanipulator of claim 1, wherein the robotic arm is provided with an electric rotating table that rotates about the x-axis, and the puncture needle and the holder are respectively provided on one of the electric rotating tables.

10. The remotely operated force feedback adaptive micromanipulator of claim 9, wherein the gripper tip is provided with a pair of jaws.