CN114848169B - Human body tracking and arranging system and method for minimally invasive surgery - Google Patents
Human body tracking and arranging system and method for minimally invasive surgery Download PDFInfo
- Publication number
- CN114848169B CN114848169B CN202210780894.5A CN202210780894A CN114848169B CN 114848169 B CN114848169 B CN 114848169B CN 202210780894 A CN202210780894 A CN 202210780894A CN 114848169 B CN114848169 B CN 114848169B
- Authority
- CN
- China
- Prior art keywords
- reflective
- reflecting
- light
- human body
- view camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2068—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis using pointers, e.g. pointers having reference marks for determining coordinates of body points
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
- A61B2090/3937—Visible markers
Abstract
The invention provides a human body tracking and arranging system and method for minimally invasive surgery, which comprises a plurality of reflective markers and a multi-view camera: the reflective mark is arranged on a human body and is arranged in the visual field range of the multi-view camera, and the orientation of the reflective mark is arranged corresponding to that of the multi-view camera; the multi-view camera identifies and tracks the reflective marks. The invention can realize rapid and convenient tracking array arrangement, is only in contact with the surface of a human body for fixation, is safe and efficient, and reduces the additional trauma brought to people by percutaneous surgical injection into the human body reference frame.
Description
Technical Field
The invention relates to the technical field of human body tracking array arrangement, in particular to a human body tracking arrangement system and a human body tracking arrangement method for minimally invasive surgery; in particular, it preferably relates to a minimally invasive surgical body tracking array deployment system.
Background
As shown in fig. 1, the conventional robotic surgery procedure is as follows: firstly, image acquisition: using CT or X-ray image acquisition (bone); operation planning: how to perform the surgery, in which bone and in which position; image registration: matching of X-ray film, CT data (including position information and other influencing data) and dual vision system (right image "optical positioning system"). Image registration may also be referred to as "image matching" at some manufacturers; fourthly, automatic positioning: the field in which automatic positioning is performed presents a significant problem of trauma, which is the "surgical" of the human reference frame for positioning; fifth, performing an operation.
As shown in fig. 2 and 3, the actual scene of the frame of reference surgery, the rigid frame of reference is placed within the field of view of the binocular camera. The rigid reference into which the array file has been entered is mounted in the field of view of a binocular camera. The binocular camera identifies the rigid reference frame by an algorithm, and under the condition of tracking the rigid body of the surgical tool, the rigid body is supposed to be composed of 4 circles, the 4 circles are guaranteed to be connected on the same plane through hardware machining, and the relative position relation among the 4 circles is known.
The reference frame is shown in fig. 2 as a portion enclosed by a dotted line. The reference frame is hit on the skeleton: (including skin shedding, bleeding): after the spinous process is exposed after the skin is opened, the spinous process clamp is used for clamping and fixing the spinous process; or the two cortical bone screw kirschner wires are fixed on the bone after percutaneous puncture. Installation is usually invasive, adding additional trauma.
As shown in FIG. 2, the reference frame is a part enclosed by a dotted line and provided with a white reflective ball (the part enclosed by the dotted line in FIG. 2 already clamps white spinal bones).
Literature references on surgical localization and related knowledge: for example, chinese patent publication No. CN102266250A discloses an ultrasonic surgical navigation system and an ultrasonic surgical navigation method, chinese patent publication No. CN109875685A discloses an orthopedic surgical navigation system and an image navigation method for orthopedic surgery, chinese patent publication No. CN109994188A discloses a neurosurgical navigation registration test method and system based on NDI, chinese patent publication No. CN113995511A discloses an actual measurement positioning surgical navigation system and a positioning surgical navigation method, chinese patent publication No. CN103356284A discloses an operative navigation method and system, chinese patent publication No. CN108210024A discloses an operative navigation method and system, chinese patent publication No. CN110650703A discloses an operative navigation system, chinese patent publication No. CN113648057A discloses an operative navigation system and an operative navigation method, Chinese patent publication No. CN113645896A discloses a system for surgical planning, surgical navigation and imaging, chinese invention patent publication No. CN112451096A discloses a method and apparatus for generating tracer identification information, chinese invention patent publication No. CN112815834B discloses an optical positioning system, chinese invention patent publication No. CN109029458B discloses a method and system for binocular visual positioning, and chinese invention patent publication No. CN109341530A discloses a method and system for object point positioning in binocular stereoscopic vision; for example, the development process and application of surgical navigation in the international journal of otorhinolaryngology and neck surgery, volume 43, year 5, month 3, and the human body mark point registration method and registration precision research in surgical navigation, volume 6, volume 25, month 12, 2008, biomedical engineering journal, can be obtained; also as binocular brief introduction.
The method comprises the following steps of performing three-dimensional scanning on the head of a patient to obtain preoperative images, obtaining a 3D model based on preoperative image processing, marking the 3D model to form marking points, printing the 3D model with the marking points attached to the 3D model to obtain a 3D printing model, and marking the 3D printing model as a body membrane; registering a coordinate system by using an optical positioning device NDI; respectively carrying out coarse registration and fine registration based on the anatomical feature points to complete preoperative intraoperative registration; and recording the coordinates of all the marking points on the body membrane by using a probe tool of NDI, and calculating the registration precision registered in the preoperative operation.
In view of the above-mentioned related art, the inventor considers that the reference frame is bumped, moved and displaced to bring about great risks: if the fixed reference frame is collided or moved in the operation, the positioned relative position fails, so that the mechanical arm is out of synchronization in the operation, and the operation is abnormal; in addition, the existing tracking arrays all need a precisely positioned reference frame for installing the reflective mark, and the size of the reference frame is large, so that the operation space (doctor operation space or mechanical arm operation space) of the operation is shielded.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a human body tracking and arranging system and method for minimally invasive surgery.
The human body tracking and arranging system for the minimally invasive surgery provided by the invention comprises a plurality of reflective markers and a multi-view camera:
the reflective mark is arranged on a human body and is arranged in the visual field range of the multi-view camera, and the orientation of the reflective mark is arranged corresponding to that of the multi-view camera;
the multi-view camera identifies and tracks the reflective marks.
Preferably, the system further comprises a reference frame for positioning the reflective marker when the reflective marker is arranged on the human body;
the reference frame is provided with an opening;
the reference frame is placed on a human body;
the reflective mark is arranged on the human body through the open hole.
Preferably, the reflective mark comprises a reflective object and a support frame;
the light-reflecting object is arranged on the support frame;
the orientation of the light-reflecting object is arranged corresponding to the multi-view camera;
the support frame is arranged on a human body.
Preferably, the multi-view camera identifies and tracks the reflective mark, and shoots the reflective mark to obtain a shot image; extracting the edge of the light-reflecting object in the shot image;
the multi-view camera back-projects the edge of the reflective object in the shot image to the space through the matrix between the internal cameras and the respective internal references of the internal cameras, and then obtains the equation of the reflective object in the space;
the multi-view camera obtains the mobile identification characteristics of the reflective marker through an equation of a reflective object in the space;
the multi-view camera judges the movement of the reflective marker through the movement recognition characteristics of the reflective marker.
Preferably, the movement recognition feature of the reflective marker comprises a central point of the reflective object and a normal of the reflective object;
the reflective marks are arranged in groups.
Preferably, when the multi-view camera identifies and tracks the grouped reflective markers, when the distance between the central points of the reflective objects is outside the set range, the reflective markers corresponding to the reflective objects outside the set range are outliers.
Preferably, when the multi-view camera identifies and tracks the grouped reflective markers, the case that the distance between the central points of the reflective objects is within the set range includes:
if the position of the central point of the light-reflecting object does not change, the grouped light-reflecting marks are judged not to move;
if the position of the central points of the light-reflecting objects changes and the distance between the central points of the light-reflecting objects does not change, the human body is judged to move integrally;
the distance between the center points of the light reflecting objects is changed and is within a set range.
Preferably, when the multi-view camera identifies and tracks the grouped reflecting marks, a normal included angle between the reflecting objects represents an orientation included angle between the reflecting objects;
and if the orientation included angle between the light-reflecting objects is outside the set orientation range, marking the light-reflecting marks outside the set orientation range as outliers.
Preferably, when the multi-view camera identifies and tracks the grouped reflective markers, the four reflective markers are grouped, and the central points of every three reflective objects are combined to form a plane respectively;
when the central points of the three light-reflecting objects are combined to form a plane, if the distance from the central point of the light-reflecting object without the formed plane to the plane is out of the set distance range, the light-reflecting mark without the formed plane is an outlier.
According to the human body tracking and arranging method for the minimally invasive surgery, which is provided by the invention, a human body tracking and arranging system for the minimally invasive surgery is applied, and the method comprises the following steps:
a mark setting step: the method comprises the following steps of arranging a light reflecting mark on a human body, wherein the light reflecting mark is arranged in the visual field range of a multi-view camera, and the orientation of the light reflecting mark is arranged corresponding to that of the multi-view camera;
and (3) identification and tracking steps: the multi-view camera identifies and tracks the reflective marks.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention can realize rapid and convenient tracking array arrangement, is only contacted and fixed with the surface of a human body, is safe and efficient, and reduces the additional trauma brought to people by percutaneous operation of striking the human body reference frame;
2. the 4 noninvasive light-reflecting markers are respectively fixed with the human body, can be self-adapted to the surface shape of the human body, and have strong applicability;
3. rigid connection is not needed among the 4 noninvasive light-reflecting markers, so that the operation space and the visual field are not interfered;
4. the noninvasive light-reflecting marker is fixed with a human body in a noninvasive mode, so that unnecessary wounds are reduced (the damage to human body tissues and skin caused by inserting a reference frame is reduced, the wounds are reduced, intraoperative hemorrhage caused by skin opening is reduced, and indexes that the operation time is reduced and intraoperative hemorrhage is reduced are improved);
5. compared with the traditional mode, the distance between the noninvasive light-reflecting marker and the human body is closer, and the probability of relative displacement is lower and the noninvasive light-reflecting marker is more reliable.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic view of a conventional robotic surgery;
FIG. 2 is a scene view of a reference frame procedure;
FIG. 3 is a schematic view of a reference frame procedure;
FIG. 4 is a layout view of a minimally invasive surgical body tracking array according to the present invention;
FIG. 5 is a schematic view of the invention highlighting the opening.
Reference numerals:
| |
|
| |
|
| Non-invasive |
|
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
According to the minimally invasive surgery human body tracking array arrangement system provided by the invention, as shown in fig. 4 and 5, the minimally invasive surgery human body tracking array arrangement system comprises a plurality of reflective markers, a reference frame and a multi-view camera. The multi-view camera includes a binocular camera, a trinocular camera or a quadrupolar camera.
The reflective mark is arranged on a human body and is arranged in the visual field range of the multi-view camera, and the orientation of the reflective mark is arranged corresponding to that of the multi-view camera. And the multi-view camera identifies and tracks the reflective mark. The orientation of the reflective marks is uniform, and the orientation of the reflective marks is opposite to that of the binocular camera. The number of the light-reflecting marks arranged on the human body is at least four. The reflective markers are arranged in an array, and the shape of the array comprises a rectangle. The reflective markers are arranged in groups, and the four reflective marker arrays arranged in groups are rectangular in shape.
The reference frame is used for positioning the light-reflecting mark when the light-reflecting mark is arranged on a human body. The reference frame is provided with an opening. The reference frame is placed on the human body. The reflective mark is arranged on the human body through the open pore. The position of the reference frame arranged on the human body and the position of the opening of the reference frame are artificially arranged.
The reflective mark comprises a reflective object and a support frame. The light-reflecting object is arranged on the support frame. The orientation of the reflecting object is arranged corresponding to the multi-view camera; the support frame is arranged on a human body. The light reflecting object is a light reflecting circular sheet. The reflective marker is a non-invasive reflective marker fixed with the human body in a non-invasive way. The non-invasive reflective markers are placed on the body in a manner that includes adhesion. The bottom of the supporting frame is provided with an adhesive sticker similar to the adhesive sticker. The reflecting surface of the reflecting object is a micron-sized glass microsphere (or a prism type, or a microsphere type, or an active light source) inside. The reflective marks are not connected with each other.
The multi-view camera identifies and tracks the reflective mark, and shoots the reflective mark to obtain a shot image; and the edge of the light-reflecting object in the shot image is extracted.
The multi-view camera back-projects the edge of the reflective object in the shot image to the space through the matrix between the internal cameras and the respective internal references of the internal cameras, and then obtains the equation of the reflective object in the space.
The multi-view camera obtains the moving identification characteristics of the reflective markers through the equation of the reflective objects in the space.
The multi-view camera judges the movement of the reflective mark through the movement identification characteristic of the reflective mark.
The movement recognition feature of the reflective marker includes a center point of the reflective object and a normal line of the reflective object. The reflective markers are arranged in groups.
According to the pinhole imaging principle:
wherein F is a projection matrix, P is an internal reference, and M is an external reference;
is composed of
Coordinates projected onto a phase plane;
coordinates of the middle point of the world coordinate system; u represents the abscissa of a point in the world coordinate system projected onto the phase plane; v represents the vertical coordinate of a certain point projected to the phase plane in the world coordinate system; x represents the X-axis coordinate of the midpoint of the world coordinate system; y stands for world seatThe coordinate of the Y axis of the middle point of the system is marked; z represents the Z-axis coordinate of the midpoint of the world coordinate system.
Now assume that there is a 2D plane H in space, where,
wherein the content of the first and second substances,
is the origin of plane H;
is the origin coordinate of the plane coordinate system plane under the world coordinate system, and T is the transposition;
representing the origin X-axis coordinate of a plane coordinate system under a world coordinate system;
representing the origin Y-axis coordinate of a plane coordinate system under a world coordinate system;
and the coordinate of the origin Z axis of the plane coordinate system plane in the world coordinate system is shown.
Wherein the content of the first and second substances,
、
two orthogonal xy axes for the plane;
， 
two orthogonal xy axes of a plane coordinate system plane under a world coordinate system;
the X-axis coordinate of one of two orthogonal xy axes of a plane coordinate system plane in a world coordinate system;
the Y-axis coordinate is the Y-axis coordinate of one of two orthogonal xy-axes of a plane coordinate system in a world coordinate system;
the Z-axis coordinate of one of two orthogonal xy axes of a plane coordinate system plane in a world coordinate system;
the coordinate of the other axis of the two orthogonal xy axes of the plane coordinate system plane in the world coordinate system is taken as the X-axis coordinate;
the coordinate of the other axis of the two orthogonal xy axes of the plane coordinate system plane in the world coordinate system is the Y-axis coordinate;
is the Z-axis coordinate of the other axis of the two orthogonal xy axes of the plane coordinate system plane in the world coordinate system.
Wherein the content of the first and second substances,
are points in the plane H that are to be considered,
coordinates after being projected to an image plane;
is the coordinate of a certain point under the plane coordinate system,
is composed of
Coordinates after projection onto an image plane;
is the X-axis coordinate of a certain point under the plane coordinate system;
is the Y-axis coordinate of a certain point under the plane coordinate system;
the horizontal coordinate is the horizontal coordinate of a certain point under the plane coordinate system after being projected to the image plane;
is a vertical coordinate after a certain point under the plane coordinate system is projected to the image plane.
All of
All on a circle with radius r.
Wherein Q represents a matrix expression form of an equation in which the matrix is a circle.
Therefore, the first and second electrodes are formed on the substrate,
wherein, the first and the second end of the pipe are connected with each other,
the result of the above matrix calculation is a mathematical symbol.
A circle in space projected onto a plane may become an ellipse or, in some special cases, a circle, the center point of which is
Comprises the following steps:
wherein, the first and the second end of the pipe are connected with each other,
the abscissa which is the center point of the ellipse or circle;
is the ordinate of the center point of the ellipse or circle.
Therefore, the first and second electrodes are formed on the substrate,
wherein the content of the first and second substances,
the third row of the F matrix.
Wherein the content of the first and second substances,Ga mathematical symbol representing the above matrix calculation.
We then get an unbiased estimated midpoint of the image plane of the spatial circle.
When the multi-view camera identifies and tracks the grouped reflecting marks, when the distance between the central points of the reflecting objects is out of the set range, the corresponding reflecting marks out of the set range are the outliers. The setting range varies depending on the size of the human body.
When the multi-view camera identifies and tracks the grouped reflecting marks, the condition that the distance between the central points of the reflecting objects is within the set range comprises the following steps: if the position of the central point of the light-reflecting object does not change, the grouped light-reflecting marks are judged not to move; if the position of the central point of the light-reflecting object changes and the distance between the central points of the light-reflecting object does not change, the human body is judged to move integrally (for example, the operation bed is translated to drive the human body to move integrally); the distance between the center points of the light reflecting objects is changed and is within a set range.
When the multi-view camera identifies and tracks the grouped reflecting marks, the normal included angle between the reflecting objects represents the orientation included angle between the reflecting objects; and if the orientation included angle between the light-reflecting objects is outside the set orientation range, marking the light-reflecting marks outside the set orientation range as outliers. The set orientation range includes 15 °
When the multi-view camera identifies and tracks the grouped reflective markers, the four reflective markers are grouped, and the central points of every three reflective objects are combined to form a plane respectively; when the central points of the three light-reflecting objects are combined to form a plane, if the distance from the central point of the light-reflecting object without the plane to the plane (the shortest distance from the point to the plane) is out of the set distance range, the light-reflecting mark without the plane is an outlier. The set distance range includes 10 mm.
The embodiment of the invention also discloses a human body tracking and arranging method for minimally invasive surgery, and as shown in fig. 4 and 5, the human body tracking and arranging method for minimally invasive surgery comprises the following steps: a mark setting step: the reflective mark is arranged on a human body and is arranged in the visual field range of the multi-view camera, and the orientation of the reflective mark is arranged corresponding to that of the multi-view camera.
And (3) identification and tracking steps: the multi-view camera identifies and tracks the reflective marks.
The identification tracking step comprises the following steps: shooting and extracting: the multi-view camera identifies and tracks the reflective mark, and shoots the reflective mark to obtain a shot image; and the edges of the light-reflecting object in the shot image are extracted.
Equation obtaining step: the multi-view camera back-projects the edge of the reflective object in the shot image to the space through the matrix between the internal cameras and the respective internal references of the internal cameras, and then obtains the equation of the reflective object in the space.
A characteristic obtaining step: the multi-view camera obtains the moving identification characteristics of the reflective markers through the equation of the reflective objects in the space.
A step of resetting judgment: the multi-view camera judges the movement of the reflective marker through the movement identification characteristics of the reflective marker; and judging whether to remove the reset reflective marks or not according to the movement of the reflective marks.
The reset judging step comprises a setting range judging step: in the set of retroreflective markers, situations arise that include: when the distance between the central points of the light-reflecting objects is within a set range, keeping the grouped light-reflecting marks; when the distance between the central points of the light-reflecting objects is out of the set range, the corresponding light-reflecting marks out of the set range are outliers; and removing outliers or grouping light-reflecting marks, and setting the light-reflecting marks on the human body again to enable the distance between the central points of the light-reflecting objects to be within a set range.
In the set distance judging step, in the grouped reflective markers, the case where the distance between the center points of the reflective objects is within the set range includes: when the position of the central point of the light-reflecting object does not change, judging that the grouped light-reflecting marks do not move, and reserving the grouped light-reflecting marks; when the position of the central point of the light-reflecting object changes and the distance between the central points of the light-reflecting object does not change, the human body is judged to move integrally (for example, the human body is driven to move integrally by the translation of the operating table), and a group of light-reflecting marks are reserved; when the distance between the center points of the reflective objects changes, the grouped reflective markers remain.
The reset judging step further comprises an orientation range judging step: in the grouped reflective markers, a normal included angle between reflective objects represents an orientation included angle between the reflective objects; if the orientation included angle between the reflective objects is within the set orientation range, retaining the grouped reflective marks; if the orientation included angle between the light reflecting objects is outside the set orientation range, the light reflecting marks outside the set orientation range are marked as outliers, the outliers are removed, the light reflecting marks are arranged on the human body again, and the orientation angle between the light reflecting objects is in the set orientation range.
The step of determining the reset further comprises a step of determining a set distance range: in the grouped reflective markers, four reflective markers are grouped, and the central points of every three reflective objects are combined to form a plane respectively; when the central points of three reflective objects are combined to form a plane, if the distance from the central point of the reflective object which does not form the plane to the plane is within a set distance range, the reflective mark which does not form the plane is reserved; if the distance from the center point of the reflecting object without the plane to the plane is out of the set distance range, the reflecting mark without the plane is an outlier, the outlier is removed, and the reflecting mark is arranged on the human body again, so that the distance from the center point of the reset reflecting object to the plane is in the set range.
The embodiment of the invention also discloses a minimally invasive surgery noninvasive human body tracking array arrangement method, which comprises a binocular camera, 4 noninvasive reflective markers and a reference frame as shown in fig. 4 and 5. The positioning and tracking system consists of a binocular camera, 4 noninvasive reflective markers and a reference frame.
The binocular camera is arranged at a proper position, so that the noninvasive reflective markers are ensured to be in a visible range, and the noninvasive reflective markers are fixed on a human body in noninvasive modes such as adhesion after being roughly positioned through the holes on the reference frame.
When the human body tracking array is used, the binocular camera is arranged, the noninvasive reflective markers are roughly positioned through the holes in the reference frame, the noninvasive reflective markers are fixed on a human body in noninvasive modes such as adhesion and the like, the reference frame is removed, the binocular camera identifies the noninvasive reflective markers through an algorithm, the array files of the noninvasive reflective markers are generated in real time, the arrangement of the human body tracking array is completed in a noninvasive mode, and the human body structure can be tracked subsequently through the array.
The binocular camera identifies the noninvasive light reflecting markers, and the method comprises the following steps of: step 1, taking images shot by two cameras (binocular cameras), and calculating the center of an approximate bright spot.
And 2, extracting a small image near the center calculated in the step 1, and extracting the edge of the small image through a sub-pixel algorithm.
And 3, fitting the edges to form an ellipse, and calculating the center of the ellipse.
And 4, matching the centers of all the ellipses left and right one by one through a matrix between the two cameras and respective internal references of the two cameras. Left-right matching is that the left side is matched with the left side, and the right side is matched with the right side, and then the distance can be known.
And 5, back projecting the sub-pixel edge calculated in the step 2 into the space through a matrix between the two cameras and respective internal references of the two cameras, and obtaining an equation of a circle in the space through matrix operation.
And 6, obtaining the central point of the circle and the normal of the circle in the space through the equation of the circle.
And 7, under the condition of tracking the rigid body of the surgical tool, supposing that the rigid body consists of 4 circles, and ensuring that the 4 circles are connected on the same plane through hardware machining and the relative position relation among the 4 circles is known.
And 8, when the circle pastes on the body surface are tracked, the circle pastes form one or a plurality of planes because the circle pastes are stuck on the body surface, and the orientation is basically uniform (not more than 10 degrees). Selecting a circle which is not fit into a rigid body of the surgical tool in space, and fitting the circle into a plane approximately, wherein if the error of the fitting plane is overlarge, the circle which is possibly selected has an outlier; if the orientation is not uniform, then the circle that may be selected has an outlier. Outliers were rejected by the RANSAC algorithm. The structure of the body surface circular patch can be obtained.
The invention relates to an arrangement method of a noninvasive human body tracking array. The method can realize rapid and convenient tracking array arrangement, is only in contact with and fixed on the surface of a human body, and is safe and efficient.
Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, units provided by the present invention as pure computer readable program code, the system and its various devices, modules, units provided by the present invention can be fully implemented by logically programming method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices, modules and units thereof provided by the invention can be regarded as a hardware component, and the devices, modules and units included in the system for realizing various functions can also be regarded as structures in the hardware component; means, modules, units for performing the various functions may also be regarded as structures within both software modules and hardware components for performing the method.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (6)
1. A minimally invasive surgery human body tracking and arranging system is characterized by comprising a plurality of reflective markers and a multi-view camera:
the light reflecting marks are arranged on a human body and are arranged in the visual field range of the multi-view camera, and the directions of the light reflecting marks are arranged corresponding to the multi-view camera;
the multi-view camera identifies and tracks the reflective marks;
the light-reflecting mark comprises a light-reflecting object and a support frame;
the light-reflecting object is arranged on the support frame;
the orientation of the reflective object is arranged corresponding to the multi-view camera;
the support frame is arranged on a human body;
the multi-view camera identifies and tracks the reflective marks, and shoots the reflective marks to obtain shot images; extracting the edge of the light-reflecting object in the shot image;
the multi-view camera back-projects the edge of the reflective object in the shot image to the space through the matrix between the internal cameras and the respective internal references of the internal cameras, and then obtains the equation of the reflective object in the space;
the multi-view camera obtains the mobile identification characteristics of the reflective marker through an equation of a reflective object in the space;
the multi-view camera judges the movement of the reflective marker through the movement identification characteristic of the reflective marker;
the movement identification characteristics of the reflective markers comprise the central points of the reflective objects and the normals of the reflective objects;
the reflective marks are arranged in groups;
when the multi-view camera identifies and tracks the grouped reflecting marks, the normal included angle between the reflecting objects represents the orientation included angle between the reflecting objects;
and if the orientation included angle between the light-reflecting objects is outside the set orientation range, marking the light-reflecting marks outside the set orientation range as outliers.
2. The minimally invasive surgical body tracking arrangement system according to claim 1, further comprising a reference frame for locating the retro-reflective markers when they are placed on the body;
the reference frame is provided with an opening;
the reference frame is placed on a human body;
the reflective mark is arranged on the human body through the open hole.
3. The minimally invasive surgery human body tracking and arranging system according to claim 1, wherein when the multi-view camera identifies and tracks the grouped retro-reflective markers, when the distance between the central points of the retro-reflective objects is out of the set range, the corresponding retro-reflective markers out of the set range are outliers.
4. The minimally invasive surgical body tracking arrangement system according to claim 3, wherein the case that the distance between the center points of the light reflecting objects is within a set range when the multi-view camera identifies and tracks the set of light reflecting markers comprises:
if the position of the central point of the light-reflecting object does not change, the grouped light-reflecting marks are judged not to move;
if the position of the central points of the light-reflecting objects changes and the distance between the central points of the light-reflecting objects does not change, the human body is judged to move integrally;
the distance between the center points of the light reflecting objects is changed and is within a set range.
5. The minimally invasive surgery human body tracking and arranging system according to claim 1, wherein when the multi-view camera identifies and tracks the grouped reflective markers, the four reflective markers are grouped, and the central points of every three reflective objects are combined to form a plane respectively;
when the central points of the three light-reflecting objects are combined to form a plane, if the distance from the central point of the light-reflecting object without the formed plane to the plane is out of the set distance range, the light-reflecting mark without the formed plane is an outlier.
6. A minimally invasive surgery human body tracking and arranging method is characterized in that the minimally invasive surgery human body tracking and arranging system of any one of claims 1-5 is applied, and comprises the following steps:
a mark setting step: the method comprises the following steps of arranging a reflective mark on a human body, wherein the reflective mark is arranged in the visual field range of a multi-view camera, and the orientation of the reflective mark is arranged corresponding to that of the multi-view camera;
and (3) identification and tracking steps: the multi-view camera identifies and tracks the reflective marks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210780894.5A CN114848169B (en) | 2022-07-05 | 2022-07-05 | Human body tracking and arranging system and method for minimally invasive surgery |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210780894.5A CN114848169B (en) | 2022-07-05 | 2022-07-05 | Human body tracking and arranging system and method for minimally invasive surgery |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114848169A CN114848169A (en) | 2022-08-05 |
| CN114848169B true CN114848169B (en) | 2022-09-30 |
Family
ID=82626739
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210780894.5A Active CN114848169B (en) | 2022-07-05 | 2022-07-05 | Human body tracking and arranging system and method for minimally invasive surgery |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114848169B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN206333963U (en) * | 2017-01-10 | 2017-07-18 | 北京积水潭医院 | Noninvasive fixing device for spinal operation navigation tracer |
| CN107330926A (en) * | 2017-08-25 | 2017-11-07 | 上海嘉奥信息科技发展有限公司 | Non-marked medical figure registration system and method in a kind of art in navigation system |
| CN108972559A (en) * | 2018-08-20 | 2018-12-11 | 上海嘉奥信息科技发展有限公司 | Hand and eye calibrating method based on infrared stereoscopic vision positioning system and mechanical arm |
| CN213098281U (en) * | 2020-04-26 | 2021-05-04 | 深圳市鑫君特智能医疗器械有限公司 | Surgical navigation system |
| CN112807086A (en) * | 2021-02-05 | 2021-05-18 | 新博医疗技术有限公司 | Operation navigation device and method based on template |
| CN113040909A (en) * | 2021-02-26 | 2021-06-29 | 张志宏 | Optical tracking system and method based on near-infrared three-eye stereo vision |
| CN113768623A (en) * | 2020-06-09 | 2021-12-10 | 上海舍成医疗器械有限公司 | Surgical navigation system using monocular positioning tracker |
| CN114373003A (en) * | 2022-01-12 | 2022-04-19 | 上海寻是科技有限公司 | Binocular vision-based passive infrared marker surgical instrument registration method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7720522B2 (en) * | 2003-02-25 | 2010-05-18 | Medtronic, Inc. | Fiducial marker devices, tools, and methods |
| CN109938839B (en) * | 2015-08-31 | 2024-03-26 | 北京天智航医疗科技股份有限公司 | Optical tracking tool for navigation surgery |
| CN114305683B (en) * | 2021-12-03 | 2023-01-06 | 哈尔滨工业大学 | Surgical instrument registration device and method |
-
2022
- 2022-07-05 CN CN202210780894.5A patent/CN114848169B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN206333963U (en) * | 2017-01-10 | 2017-07-18 | 北京积水潭医院 | Noninvasive fixing device for spinal operation navigation tracer |
| CN107330926A (en) * | 2017-08-25 | 2017-11-07 | 上海嘉奥信息科技发展有限公司 | Non-marked medical figure registration system and method in a kind of art in navigation system |
| CN108972559A (en) * | 2018-08-20 | 2018-12-11 | 上海嘉奥信息科技发展有限公司 | Hand and eye calibrating method based on infrared stereoscopic vision positioning system and mechanical arm |
| CN213098281U (en) * | 2020-04-26 | 2021-05-04 | 深圳市鑫君特智能医疗器械有限公司 | Surgical navigation system |
| CN113768623A (en) * | 2020-06-09 | 2021-12-10 | 上海舍成医疗器械有限公司 | Surgical navigation system using monocular positioning tracker |
| CN112807086A (en) * | 2021-02-05 | 2021-05-18 | 新博医疗技术有限公司 | Operation navigation device and method based on template |
| CN113040909A (en) * | 2021-02-26 | 2021-06-29 | 张志宏 | Optical tracking system and method based on near-infrared three-eye stereo vision |
| CN114373003A (en) * | 2022-01-12 | 2022-04-19 | 上海寻是科技有限公司 | Binocular vision-based passive infrared marker surgical instrument registration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114848169A (en) | 2022-08-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11801097B2 (en) | Robotic fluoroscopic navigation | |
| US20200323609A1 (en) | Method and system for improving 2d-3d registration convergence | |
| EP3306567B1 (en) | System for improving 2d-3d registration convergence | |
| EP3247305B1 (en) | Dynamic reference arrays | |
| US10102640B2 (en) | Registering three-dimensional image data of an imaged object with a set of two-dimensional projection images of the object | |
| US6275725B1 (en) | Stereotactic optical navigation | |
| CN111388087A (en) | Surgical navigation system, computer and storage medium for performing surgical navigation method | |
| US9833291B2 (en) | Ultrasound CT registration for positioning | |
| US6856826B2 (en) | Fluoroscopic tracking and visualization system | |
| US20210153953A1 (en) | Systems and methods for performing intraoperative guidance | |
| US20030088179A1 (en) | Fluoroscopic tracking and visualization system | |
| CN111714206A (en) | Neuro-navigation registration and robot trajectory guidance system and related method and device | |
| US20220142730A1 (en) | Fiducial marker | |
| US20230329799A1 (en) | Rotating Marker | |
| CN101616633A (en) | Reference mark is arranged | |
| EP3295887B1 (en) | Robotic fluoroscopic navigation | |
| WO2021019369A1 (en) | Fiducial marker | |
| CN113491578A (en) | Method for registering medical images to a circle-arc assembly | |
| EP3733112A1 (en) | System for robotic trajectory guidance for navigated biopsy needle | |
| CN114848169B (en) | Human body tracking and arranging system and method for minimally invasive surgery | |
| CN213098281U (en) | Surgical navigation system | |
| TWM605788U (en) | Positioning system | |
| Wei et al. | Multi-Camera AR Navigation System for CT-Guided Needle Insertion Task | |
| US20240074819A1 (en) | Robotic fluoroscopic navigation | |
| US20210401510A1 (en) | Dynamic Reference Arrays and Methods of Use |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |