CN113040909A - Optical tracking system and method based on near-infrared three-eye stereo vision - Google Patents
Optical tracking system and method based on near-infrared three-eye stereo vision Download PDFInfo
- Publication number
- CN113040909A CN113040909A CN202110220767.5A CN202110220767A CN113040909A CN 113040909 A CN113040909 A CN 113040909A CN 202110220767 A CN202110220767 A CN 202110220767A CN 113040909 A CN113040909 A CN 113040909A
- Authority
- CN
- China
- Prior art keywords
- surgical instrument
- infrared
- optical
- cameras
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 77
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000008569 process Effects 0.000 claims description 11
- 230000000712 assembly Effects 0.000 claims description 10
- 238000000429 assembly Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000009434 installation Methods 0.000 abstract description 3
- 238000013461 design Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 210000000214 mouth Anatomy 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 239000003550 marker Substances 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000007943 implant Substances 0.000 description 2
- 238000002513 implantation Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 229920000049 Carbon (fiber) Polymers 0.000 description 1
- 238000012331 Postoperative analysis Methods 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000004917 carbon fiber Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2055—Optical tracking systems
- A61B2034/2057—Details of tracking cameras
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Robotics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses an optical tracking system and method based on near-infrared three-eye stereo vision, belonging to the technical field of medical instruments. The three cameras pick up images from the oblique upper part, and a form of combining the cameras and the plane mirror is adopted, so that a larger baseline distance can be obtained in the same size space, the layout is more compact, the cameras are positioned on the same plane, and the installation is more convenient; the view field of the camera can be adjusted by adjusting the angle of the reflector; still install miniature laser instrument at the top of camera, can confirm the angle intersection position of camera optical axis in space fast before measuring, can estimate out the focus area of each camera in space directly perceived, conveniently place the target object, be worth using widely.
Description
Technical Field
The invention relates to the technical field of medical instruments, in particular to an optical tracking system and method based on near-infrared three-eye stereoscopic vision.
Background
The optical tracking system mainly comprises a multi-view stereoscopic vision platform and an operation target. In order to eliminate the influence of ambient light on the extraction of the marker points, the system often operates in the near infrared band. The calibrated multiple cameras shoot the target from different angles, and the distances between each marking point and the working point are calibrated in advance because the surgical target is a rigid body, so that the position of the working point can be obtained by calculation. The method can be used in the field that a plurality of working points are blocked or difficult to measure. For example, in the oral implant operation, a doctor needs to use an implant mobile phone to drill a hole at the edentulous position of a patient, the sight of the doctor is very easy to be blocked, and the depth and the angle of the drilled hole are often judged according to the experience of the doctor. By adopting the optical tracking system, the optical mark points can be arranged on the mobile phone, the position of the drill bit of the mobile phone can be obtained by tracking the optical mark points, and the drill bit can be positioned and tracked in real time.
Since optical tracking systems have important roles in many fields, a large number of researchers have made their contributions in the research of optical navigation systems in the past decades.
Early optical tracking systems were mainly applied in the industrial field, and since the last 90 s, optical navigation systems were gradually applied in the medical field with the development of computer-aided technology and optical positioning technology.
In 1990, the first infrared surgical navigation system, the Stealthstation System, was introduced by Medtronic Inc., and the development of surgical navigation entered the blowout period. At present, representative foreign institutions in the field of surgical navigation include computer-integrated surgical systems and technical groups of the national science foundation engineering center of hopkins university, image navigation laboratories of stanford university, medical robots of camibo merlon university and siben farina hospital in cooperation with computer-assisted surgical research center, the science of science and institute of technology, and the medical vision group of artificial intelligence laboratories, and the like. Many foreign medical device companies are also competing for surgical navigational development, such as toshiba, medtronic, siemens, philips, general medicine, and so forth. Most products on the market today are optical surgical navigation systems. Wherein the Polaris system is considered as an international standard for navigators, and the Stealthstation system is the only surgical navigation system that can be used for the whole body by FDA certification. They are two major products of the current surgical navigation system and represent the highest level of the surgical navigation system internationally. Taking the polar system of NDI as an example, the current state of the art is: the measurement precision is 0.35 mmRMS; can track 9 passive surgical instruments and 3 active surgical instruments at the same time; the maximum data update rate is 60 Hz. The research on the surgical navigation system in China is relatively late. The units engaged in this study were mainly concentrated in Shanghai transportation university, Compound denier university, Beijing aerospace university, Qinghua university, Beijing university, Zhejiang university, south China university, southeast university, Anke corporation, and Shanghai Yinjiao Equipment Limited. The neurosurgery of the Huashan Hospital affiliated to the university of Compound Dan introduced the first surgical navigation device in 1997; the first home-made surgical navigation system was introduced by the department of ann in 1999; the digital medical research center of the university of fudan introduced a high-precision neurosurgical navigation system in 2006. These products are far from the international advanced level, both in terms of precision and in terms of intelligence. In addition, the research direction of domestic institutions engaged in research mainly focuses on theory, and a high-precision dynamic navigator which can be really applied to actual surgical operations is lacked. Therefore, the research and development of the high-precision intelligent oral implantation navigator has important practical value for improving the theory and equipment level of the domestic medical navigator, improving the accuracy and safety of oral implantation and meeting the continuously improved oral medical requirements of Chinese people.
The existing optical tracking system has two problems, one is low precision, the main manufacturer of the optical tracking system at the present stage is a dynamic navigation system of Canadian NDI Polaris, the precision can reach 0.35mm, but the precision is difficult to further improve due to the limitation of binocular vision; the second problem is occlusion, and once a certain camera is occluded or an optical mark point is not in the common visual field, the whole system cannot work in the existing optical tracking system. Therefore, an optical tracking system and method based on near-infrared three-eye stereo vision are provided.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: how to solve the problems of shielding and the like of the existing optical tracking system and provide an optical tracking system based on near-infrared three-eye stereo vision.
The invention solves the technical problems through the following technical scheme, and the device comprises a camera calibration module, a data acquisition module, a three-dimensional reconstruction module, a surgical instrument reconstruction module, a data reading and storing module and a real-time display and navigation module;
the camera calibration module is used for calibrating the near-infrared trinocular visual platform;
the data acquisition module is used for acquiring image information;
the three-dimensional reconstruction module is used for three-dimensionally reconstructing the spatial position of the optical mark point on the surgical instrument;
the surgical instrument reconstruction module is used for identifying the surgical instrument according to the European distance characteristics of the surgical instrument; performing three-dimensional reconstruction on the surgical instrument by using the structural characteristics of the surgical instrument and the position of the marking point of the surgical instrument to obtain the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument;
the data reading and storing module is used for loading oral jaw model data, surgical instrument model data and other auxiliary instrument model data and storing calculation data;
and the real-time display and navigation module is used for updating the position and the posture of the surgical instrument in the software interface in real time and recording data in real time.
Furthermore, the optical mark point includes two types, namely an active type and a passive type, the passive type is a reflective ball, and the active type is an infrared light emitting diode component.
Further, the spot center in the image corresponds to the optical mark point.
Furthermore, at least three infrared light emitting diode assemblies are arranged on the surgical instrument, infrared light emitted by the infrared light emitting diode assemblies is collected by a camera, the posture of the surgical instrument is obtained by utilizing the position of the center of a light spot formed by the infrared light emitting diode assemblies in an image, and the position of the working point is obtained by utilizing the relation between the working point and the mark point.
The invention also provides an optical tracking method based on near-infrared three-eye stereo vision, which utilizes the tracking system to track the position and the posture of the surgical instrument in real time and comprises the following steps:
s1: shooting the same target by using the left, middle and right three cameras, performing self-adaptive threshold segmentation on the acquired image, solving a mark point light spot by using a mode of searching a connected domain, calculating a light spot center and performing distortion correction according to a calibration result;
s2: matching the left, middle and right image mark points, and solving the three-dimensional coordinates of the matched mark points in the left camera coordinate system according to the internal and external parameters of the left, middle and right cameras;
s3: when the surgical instrument model data is imported into a computer, registering the surgical instrument by using the mark points; in the real-time tracking process of the surgical instrument, identifying the surgical instrument through the European distance characteristics of the surgical instrument; the system realizes the real-time positioning of the marking points on the surgical instruments and calculates the rotation and translation matrixes of the surgical instruments; and obtaining the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument by using the model data of the surgical instrument and the position information of the marking point of the surgical instrument, and updating the position and the posture of the surgical instrument in a software interface in real time.
Furthermore, in the calibration process of the preoperative surgical instrument, the coordinates of all the mark points on the surgical instrument are measured, and the Euclidean distance under the world coordinate system between all the mark points is calculated to be used as the structural characteristic of the surgical instrument.
Furthermore, the calibrated world coordinate system is under the left camera coordinate system.
Compared with the prior art, the invention has the following advantages: according to the optical tracking system based on the near-infrared three-eye stereoscopic vision, three cameras pick up images from the oblique upper side, a form of combining the cameras and the plane mirror is adopted, a larger baseline distance can be obtained in the same size space, the layout is more compact, the cameras are located on the same plane, and the installation is more convenient; the view field of the camera can be adjusted by adjusting the angle of the reflector; still install miniature laser instrument at the top of camera, can confirm the angle intersection position of camera optical axis in space fast before measuring, can estimate out the focus area of each camera in space directly perceived, conveniently place the target object, be worth using widely.
Drawings
Fig. 1 is a block diagram of a communication structure in a second embodiment of the present invention;
FIG. 2 is a schematic diagram of the positions of three cameras according to a second embodiment of the present invention;
FIG. 3 is a flowchart of software processing according to a second embodiment of the present invention;
FIG. 4 is a schematic diagram of the synchronous processing of the camera according to the second embodiment of the present invention;
fig. 5 is a software interface diagram in the third embodiment of the present invention.
In fig. 2: 1. a plane mirror; 2. a lens; 3. a laser; 4. a signal trigger interface; 5. a data transmission interface; 6. a camera; 7. a filter plate;
in fig. 4: 21. a camera; 22. PS and PL; 23. and (4) an upper computer.
Detailed Description
The following examples are given for the detailed implementation and specific operation of the present invention, but the scope of the present invention is not limited to the following examples.
Example one
The embodiment provides a technical scheme: an optical tracking system based on near-infrared three-eye stereo vision comprises a camera calibration module, a data acquisition module, a three-dimensional reconstruction module, a surgical instrument reconstruction module, a data reading and storing module and a real-time display and navigation module;
the camera calibration module is used for calibrating the near-infrared trinocular visual platform;
the data acquisition module is used for acquiring image data;
the three-dimensional reconstruction module is used for three-dimensionally reconstructing the spatial position of the optical marking point on the surgical instrument;
the surgical instrument reconstruction module is used for identifying the surgical instrument according to the European distance characteristics of the surgical instrument; performing three-dimensional reconstruction on the surgical instrument by using the structural characteristics of the surgical instrument and the position of the marking point of the surgical instrument to obtain the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument;
the data reading and storing module is used for loading oral jaw model data, surgical instrument model data and other auxiliary instrument model data and storing calculation data;
and the real-time display and navigation module is used for updating the position and the posture of the surgical instrument in the software interface in real time and recording data in real time.
In this embodiment, the near-infrared trinocular visual platform includes three cameras and three reflectors, the three cameras are installed on the same plane and located obliquely above the operation position, the three reflectors are respectively arranged at the lens of the cameras, and the viewing field area of the cameras is adjusted by adjusting the angles of the reflectors.
In this embodiment, the surgical instrument is provided with at least three infrared light emitting diode assemblies, infrared light emitted by the infrared light emitting diode assemblies is collected by the camera, the posture of the surgical instrument is obtained by using the positions of the centers of light spots formed by the infrared light emitting diode assemblies in the image, and the positions of the working points are obtained by using the relationship between the working points and the mark points, i.e., the centers of the light spots formed by the infrared light emitting diode assemblies in the image.
The embodiment also provides an optical tracking method based on near-infrared three-eye stereo vision, which utilizes the tracking system to track the position and the posture of the surgical instrument in real time and comprises the following steps:
s1: shooting the same target by using the left, middle and right three cameras, performing self-adaptive threshold segmentation on the acquired image, solving a mark point light spot by using a mode of searching a connected domain, calculating a light spot center and performing distortion correction according to a calibration result;
s2: matching the left, middle and right image mark points, and solving the three-dimensional coordinates of the matched mark points in the left camera coordinate system according to the internal and external parameters of the left, middle and right cameras;
s3: when the surgical instrument model data is imported into a computer, registering the surgical instrument by using the mark points; in the real-time tracking process of the surgical instrument, identifying the surgical instrument through the European distance characteristics of the surgical instrument; the system realizes the real-time positioning of the marking points on the surgical instruments and calculates the rotation and translation matrixes of the surgical instruments; and obtaining the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument by using the model data of the surgical instrument and the position information of the marking point of the surgical instrument, and updating the position and the posture of the surgical instrument in a software interface in real time.
In this embodiment, during the calibration process of the preoperative surgical instrument, the coordinates of each marker point on the surgical instrument are measured, and the euclidean distance in the world coordinate system between each marker point is calculated as the structural feature of the surgical instrument.
In this embodiment, the world coordinate system after calibration is set under the left camera coordinate system.
Example two
The optical tracking system of the present embodiment mainly includes the following design contents:
1. hardware system design
1.1 data synchronization and real-time processing design
In the design of a hardware system, data synchronization and real-time processing are basic requirements. Three-eye stereoscopic vision platform, three cameras shoot a plurality of targets at the same time, real-time processing and model rendering are required, and if hardware acceleration is not carried out, the operation cannot be carried out by a common engineering machine; the three cameras collect simultaneously, can adopt external triggering, and need to provide external clock pulses. The Xilinx extensible processing platform Zynq series of modules is the preferred choice. The camera transmits data with the FPGA chip through the USB3.0 interface, and the FPGA chip is used for hardware acceleration. A block diagram of the communication structure is shown in fig. 1.
In fig. 1, PS is a Processing System (Processing System), in this embodiment, an FPGA chip; PL, Programmable Logic (Programmable Logic), in this embodiment a Xilinx extensible processing platform Zynq series module; AXI is called advanced extensibility interface, which is an interface protocol introduced by Xilinx from 6-series FPGA, and mainly describes a data transmission mode between a master device and a slave device.
1.2 optical System design
In the design of a hardware system, the optical system is mainly designed, and the optical system comprises a camera, a light-emitting diode (a target forming core), a filter and a reflector (a plane mirror), so that the spectral characteristics of the camera, the light-emitting diode (a target forming core), the filter and the reflector are matched with each other, background light interference is removed, good image quality is obtained, and the accuracy of an optical tracking system is improved. Stereoscopy is the process of restoring picture 2D information to 3D information. Wherein depth direction information is recovered by parallax, so the longer the baseline, the higher the depth direction reconstruction accuracy. Under the premise of a narrow base line, the longer the base line is, the larger the overall size of the equipment is, and the more easily the equipment is shielded. As shown in fig. 2, in order to further solve the occlusion problem, three cameras take images from obliquely above during the actual operation, and for the purpose of optimizing the design, a combination of a camera and a plane mirror is adopted. The three cameras can obtain larger baseline distance in the same size space, the layout of the three cameras is more compact, the cameras are conveniently arranged on the same plane, and the view field area of the cameras can be adjusted by adjusting the angle of the reflector. And each shell in the three-eye stereoscopic vision platform is made of a carbon fiber material with a small thermal expansion coefficient, so that the influence of baseline change caused by temperature on precision is reduced.
2. Software system design
In the control software design phase, the design is carried out in a top-down mode. Corresponding functional areas must be divided, functions correspond to modules one by one, independence of each module is guaranteed, and meanwhile sharing of information among the modules and hierarchical relation among the modules also need to be guaranteed.
The module includes: a camera calibration module; a data acquisition module; a three-dimensional reconstruction module; a surgical instrument reconstruction module; a data reading and storing module; and the real-time display and navigation module. The main contents are as follows: compiling an interface; a near-infrared multi-camera calibration algorithm; a left, middle and right image acquisition and preprocessing algorithm; identifying optical mark points and performing a three-dimensional reconstruction algorithm; a surgical instrument segmentation algorithm; registration algorithm of preoperative surgical instruments; and (3) an algorithm for identifying surgical instruments and updating the position and the posture during operation.
As shown in fig. 3, the left, middle and right three cameras shoot the same target, median filtering is performed on the collected image, gray information is counted, a proper threshold is calculated, adaptive threshold segmentation is performed, a light spot of an optical mark point in the image is obtained by searching a connected domain, the center of the light spot is calculated, distortion of the image is corrected according to a calibration result of the trinocular vision camera, then matching of the left, middle and right image optical mark points is performed, and a three-dimensional coordinate (a left camera coordinate system is a calibrated world coordinate system) of the matched optical mark point in a left camera coordinate system is obtained according to internal and external parameters of the left, middle and right cameras.
In the calibration process of the preoperative surgical instrument, the coordinates of optical mark points on the surgical instrument are measured, and the Euclidean distance under a world coordinate system among the optical mark points is calculated to be used as the structural characteristic of the surgical instrument.
When the surgical instrument model data is imported into a computer, registering the surgical instrument by using the optical mark points; in the real-time tracking process of the surgical instrument, identifying the surgical instrument through the European distance characteristics of the surgical instrument; the trinocular vision system realizes the real-time positioning of the optical mark points on the surgical instruments and can calculate the rotation and translation matrix of the surgical instruments; because the surgical instrument is a rigid body, the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument can be obtained by utilizing the model data of the surgical instrument and the position information of the optical marking point of the surgical instrument. And updating the position and the posture of the surgical instrument in the software interface in real time.
The optical mark points of the surgical instrument comprise an active type and a passive type, wherein the passive type is a reflective ball, and the active type is an LED.
The operation navigation part projects each image under the same coordinate system through three-dimensional image registration, the three-dimensional image of the oral cavity model is provided with optical mark points, and the position and the motion track of the motion of the surgical instrument in the oral cavity can be displayed in real time in a computer screen through matching the optical mark points in the actual space and the three-dimensional image in the operation.
In the operation navigation process, various navigation parameters including the position and the posture of the surgical instrument can be checked in real time, and data can be recorded in real time, so that the operation navigation process is convenient for postoperative analysis.
The optical tracking system based on the trinocular stereo vision is a complete system, consists of a trinocular stereo vision platform, an optical target and control software, can independently complete the functions of real-time tracking, displaying, recording and the like of the optical target, can be combined with surgical instruments to replace the eyes of doctors, displays the positions and the directions of the surgical instruments and the anatomical structures of focus positions in real time, and reduces the surgical trauma to the maximum extent. Has wide application prospect in the fields of surgical operation, robot, etc.
EXAMPLE III
A series of experiments were performed using an optical tracking system based on near-infrared trinocular stereo vision and a model of the jaw bone of the mouth (designed with 15 marker points). The experiments were exemplified with the model of the oral jaw.
The following is a description of the operation:
1. placing equipment in a proper area, placing a marking plate at the front end, setting camera parameters, clicking to start acquisition, acquiring a series of near-infrared calibration plate samples, and calibrating a trinocular vision system by adopting an MATLAB Zhang calibration package;
2. solving a basic matrix between the cameras by using a self-programming program;
3. clicking 'camera parameter setting', and filling a calibration result into self-editing software;
4. selecting a 'planting mobile phone 1', rotating a target sharp point around a certain point, shooting five positions, and finishing target registration;
5. the main interface starts to display the model of the mobile planting phone 1, the position and the posture are displayed in real time, and the coordinates of the target sharp point can be read out in a data display frame;
6. the target sharp point clicks a pit on the jaw bone of the oral cavity, the target sharp point is placed in the pit, and the 'registration' is clicked once on a software interface. It should be noted that the jaw bone is fixed and the order of clicking the pits needs to be consistent with the order noted in the program. After five clicks, the oral jaw registration is completed, and the jaw model appears on a display interface;
7. and clicking the mark point on the jaw bone of the oral cavity again by the target sharp point, and recording the coordinate and the deviation by software to complete the positioning experiment.
It should be noted that the measurement of the position and posture of the jaw bone in the oral cavity can also be calculated by optical mark points fixed on the jaw bone. At present, the oral jaw clamping device, the universal connecting rod and the reference plate are not adapted, so that the position and the posture of the oral jaw under the world coordinate system are obtained in the mode.
To sum up, in the optical tracking system based on near-infrared three-eye stereo vision of the above embodiment, three cameras take images from obliquely above, and a form of combining the cameras and the plane mirror is adopted, so that a larger baseline distance can be obtained in the same size space, the layout is more compact, the cameras are located on the same plane, and the installation is more convenient; the view field of the camera can be adjusted by adjusting the angle of the reflector; still install miniature laser instrument at the top of camera, can confirm the angle intersection position of camera optical axis in space fast before measuring, can estimate out the focus area of each camera in space directly perceived, conveniently place the target object, be worth using widely.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (8)
1. An optical tracking system based on near-infrared three-eye stereo vision is characterized by comprising a camera calibration module, a data acquisition module, a three-dimensional reconstruction module, a surgical instrument reconstruction module, a data reading and storing module and a real-time display and navigation module;
the camera calibration module is used for calibrating the near-infrared trinocular visual platform;
the data acquisition module is used for acquiring image information;
the three-dimensional reconstruction module is used for three-dimensionally reconstructing the spatial position of the optical mark point on the surgical instrument;
the surgical instrument reconstruction module is used for identifying the surgical instrument according to the European distance characteristics of the surgical instrument; performing three-dimensional reconstruction on the surgical instrument by using the structural characteristics of the surgical instrument and the position of the optical mark point of the surgical instrument to obtain the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument;
the data reading and storing module is used for loading affected part model data and storing calculation data;
and the real-time display and navigation module is used for updating the position and the posture of the surgical instrument in the software interface in real time and recording data in real time.
2. The optical tracking system based on near infrared trinocular stereovision as claimed in claim 1, wherein: the near-infrared trinocular visual platform comprises three cameras and three reflectors, the three cameras are arranged on the same plane and are located obliquely above an operation position, the three reflectors are respectively arranged at the positions of lenses of the cameras, and the view field area of the cameras is adjusted by adjusting the angles of the reflectors.
3. The optical tracking system based on near infrared trinocular stereovision as claimed in claim 2, wherein: the center of the light spot in the image corresponds to the optical mark point.
4. The optical tracking system based on near infrared trinocular stereovision as claimed in claim 3, wherein: the optical mark point comprises an active type and a passive type, wherein the passive type is a reflective ball, and the active type is an infrared light emitting diode component.
5. The optical tracking system of claim 4, wherein the optical tracking system is based on near infrared trinocular stereo vision: the surgical instrument is provided with at least three infrared light emitting diode assemblies, infrared light emitted by the infrared light emitting diode assemblies is collected by a camera, the posture of the surgical instrument is obtained by utilizing the position of the center of a light spot formed by the infrared light emitting diode assemblies in an image, and the position of a working point is obtained by utilizing the relation between the working point and an optical mark point.
6. An optical tracking method based on near-infrared trinocular stereovision, which is characterized in that the position and the posture of a surgical instrument are tracked in real time by using the tracking system of any one of claims 1 to 5, and the method comprises the following steps:
s1: shooting the same target by using three cameras on the left, the middle and the right, performing self-adaptive threshold segmentation on the acquired image, solving a light spot of an optical mark point by using a mode of searching a connected domain, calculating the center of the light spot and performing distortion correction according to a calibration result;
s2: matching the left, middle and right image optical mark points, and solving the three-dimensional coordinates of the matched optical mark points in the left camera coordinate system according to the internal and external parameters of the left, middle and right cameras;
s3: when the surgical instrument model data is imported into a computer, registering the surgical instrument by using the optical mark points; in the real-time tracking process of the surgical instrument, identifying the surgical instrument through the European distance characteristics of the surgical instrument; the system realizes the real-time positioning of the optical mark points on the surgical instruments and calculates the rotation and translation matrix of the surgical instruments; and obtaining the three-dimensional space position of the working point of the surgical instrument and the posture of the surgical instrument by using the model data of the surgical instrument and the position information of the optical mark point of the surgical instrument, and updating the position and the posture of the surgical instrument in a software interface in real time.
7. The optical tracking method based on near-infrared trinocular stereovision according to claim 6, characterized in that: in the calibration process of the preoperative surgical instrument, the coordinates of all optical mark points on the surgical instrument are measured, and the Euclidean distance under a world coordinate system between all the optical mark points is calculated and used as the structural characteristic of the surgical instrument.
8. The optical tracking method based on near-infrared trinocular stereovision according to claim 7, characterized in that: and the left camera coordinate system is a calibrated world coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110220767.5A CN113040909A (en) | 2021-02-26 | 2021-02-26 | Optical tracking system and method based on near-infrared three-eye stereo vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110220767.5A CN113040909A (en) | 2021-02-26 | 2021-02-26 | Optical tracking system and method based on near-infrared three-eye stereo vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113040909A true CN113040909A (en) | 2021-06-29 |
Family
ID=76509239
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110220767.5A Pending CN113040909A (en) | 2021-02-26 | 2021-02-26 | Optical tracking system and method based on near-infrared three-eye stereo vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113040909A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114848169A (en) * | 2022-07-05 | 2022-08-05 | 上海嘉奥信息科技发展有限公司 | Human body tracking and arranging system and method for minimally invasive surgery |
| CN115359048A (en) * | 2022-10-19 | 2022-11-18 | 中国工程物理研究院应用电子学研究所 | Real-time dynamic alignment measurement method based on closed-loop tracking and aiming and tracking and aiming device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101214142A (en) * | 2007-12-26 | 2008-07-09 | 北京理工大学 | Medicine operation navigation tracking device based on multispectral image |
| CN201917461U (en) * | 2011-01-18 | 2011-08-03 | 吉林大学 | Small-size long-base-line-distance stereoscopic vision detecting system for automobile wheel alignment parameter |
| US20180068441A1 (en) * | 2014-07-23 | 2018-03-08 | Kineticor, Inc. | Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan |
| CN107874832A (en) * | 2017-11-22 | 2018-04-06 | 合肥美亚光电技术股份有限公司 | Bone surgery set navigation system and method |
| CN109099838A (en) * | 2018-08-16 | 2018-12-28 | 华南农业大学 | A kind of single eye stereo vision system three-dimensional information measurement method based on catadioptric |
| CN109211198A (en) * | 2018-08-15 | 2019-01-15 | 河海大学常州校区 | A kind of intelligent Target detection and measuring system and method based on trinocular vision |
| CN111402309A (en) * | 2020-03-05 | 2020-07-10 | 中国科学院上海微***与信息技术研究所 | Calibration and depth image processing method for multi-view stereo camera |
-
2021
- 2021-02-26 CN CN202110220767.5A patent/CN113040909A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101214142A (en) * | 2007-12-26 | 2008-07-09 | 北京理工大学 | Medicine operation navigation tracking device based on multispectral image |
| CN201917461U (en) * | 2011-01-18 | 2011-08-03 | 吉林大学 | Small-size long-base-line-distance stereoscopic vision detecting system for automobile wheel alignment parameter |
| US20180068441A1 (en) * | 2014-07-23 | 2018-03-08 | Kineticor, Inc. | Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan |
| CN107874832A (en) * | 2017-11-22 | 2018-04-06 | 合肥美亚光电技术股份有限公司 | Bone surgery set navigation system and method |
| CN109211198A (en) * | 2018-08-15 | 2019-01-15 | 河海大学常州校区 | A kind of intelligent Target detection and measuring system and method based on trinocular vision |
| CN109099838A (en) * | 2018-08-16 | 2018-12-28 | 华南农业大学 | A kind of single eye stereo vision system three-dimensional information measurement method based on catadioptric |
| CN111402309A (en) * | 2020-03-05 | 2020-07-10 | 中国科学院上海微***与信息技术研究所 | Calibration and depth image processing method for multi-view stereo camera |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114848169A (en) * | 2022-07-05 | 2022-08-05 | 上海嘉奥信息科技发展有限公司 | Human body tracking and arranging system and method for minimally invasive surgery |
| CN114848169B (en) * | 2022-07-05 | 2022-09-30 | 上海嘉奥信息科技发展有限公司 | Human body tracking and arranging system and method for minimally invasive surgery |
| CN115359048A (en) * | 2022-10-19 | 2022-11-18 | 中国工程物理研究院应用电子学研究所 | Real-time dynamic alignment measurement method based on closed-loop tracking and aiming and tracking and aiming device |
| CN115359048B (en) * | 2022-10-19 | 2023-01-31 | 中国工程物理研究院应用电子学研究所 | Real-time dynamic alignment measurement method based on closed-loop tracking and aiming and tracking and aiming device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tuceryan et al. | Single-point active alignment method (SPAAM) for optical see-through HMD calibration for augmented reality | |
| KR102373714B1 (en) | Quantitative three-dimensional imaging of surgical scenes from multiport perspectives | |
| Deacon et al. | The Pathfinder image-guided surgical robot | |
| CN108227929B (en) | Augmented reality lofting system based on BIM technology and implementation method | |
| CN108022302B (en) | Stereo display device of Inside-Out space orientation's AR | |
| CN113876426B (en) | Intraoperative positioning and tracking system and method combined with shadowless lamp | |
| CN113040909A (en) | Optical tracking system and method based on near-infrared three-eye stereo vision | |
| US8704827B2 (en) | Cumulative buffering for surface imaging | |
| CN204863476U (en) | Bone surgery positioning system | |
| CN105496556B (en) | A kind of high-precision optical positioning system for surgical navigational | |
| CN114668534B (en) | Intraoperative implantation precision detection system and method for dental implant surgery | |
| CN112215871B (en) | Moving target tracking method and device based on robot vision | |
| TWI697317B (en) | Digital image reality alignment kit and method applied to mixed reality system for surgical navigation | |
| CN101224104A (en) | Orthopaedics operation navigation system | |
| Maurer Jr et al. | Augmented-reality visualization of brain structures with stereo and kinetic depth cues: system description and initial evaluation with head phantom | |
| CN113208731A (en) | Binocular vision system-based hand and eye calibration method for surgical puncture robot | |
| CN113034700A (en) | Anterior cruciate ligament reconstruction surgery navigation method and system based on mobile terminal | |
| CN113925441B (en) | Imaging method and imaging system based on endoscope | |
| CN111658142A (en) | MR-based focus holographic navigation method and system | |
| Wang et al. | Real-time marker-free patient registration and image-based navigation using stereovision for dental surgery | |
| Deacon et al. | Evaluation of a CCD-based facial measurement system | |
| US20030179249A1 (en) | User interface for three-dimensional data sets | |
| CN114004880B (en) | Point cloud and strong reflection target real-time positioning method of binocular camera | |
| CN112294453B (en) | Microsurgery surgical field three-dimensional reconstruction system and method | |
| AU2021376535A1 (en) | Scanner for intraoperative application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210629 |